Re: Doc tweak for huge_pages? - Mailing list pgsql-hackers

From Catalin Iacob
Subject Re: Doc tweak for huge_pages?
Date
Msg-id CAHg_5goYgh_1G8Z7QEnjemgaBObkrzV1++_ne6yrRVxQfH91ew@mail.gmail.com
Whole thread Raw
In response to Re: Doc tweak for huge_pages?  (Catalin Iacob <iacobcatalin@gmail.com>)
Responses Re: Doc tweak for huge_pages?
List pgsql-hackers
On Tue, Jan 23, 2018 at 7:13 PM, Catalin Iacob <iacobcatalin@gmail.com> wrote:
> By the way, Fedora 27 does disable THP by default, they deviate from
> upstream in this regard:

> When I have some time I'll try to do some digging into history of the
> Fedora kernel package to see if they provide a rationale for changing
> the default. That might hint whether it's likely that future RHEL will
> change as well.

I see Peter assigned himself as committer, some more information below
for him to decide on the strength of the anti THP message.

commit 9a031d5070d9f8f5916c48637bd0c237cd52eaf9
Author: Josh Boyer <jwboyer@redhat.com>
Date:   Thu Mar 27 18:31:16 2014 -0400

    Switch to CONFIG_TRANSPARENT_HUGEPAGE_MADVISE instead of always on

    The benefit of THP has been somewhat questionable overall for a while,
    and it's been known to cause performance issues with some workloads.
    Upstream also considers it to be overly complicated and really not worth
    it on machines with memory in the amounts found on typical desktops/SMB
    servers.

    Switch to using it via madvise, which most applications that care about
    it should likely already be doing.

Debian 9 also seems to default to madvise instead of always.

Digging more into it, there were changes in the 4.6 kernel (released
May 2016) that should improve THP, more precisely:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=444eb2a449ef36fe115431ed7b71467c4563c7f1

This also lead Debian to change their default in September 2017 (so
for the future Debian release) back to always, referencing the 44eb2a
improvements:
https://anonscm.debian.org/cgit/kernel/linux.git/commit/debian/changelog?id=611a8e67260e8b8190ab991206a3867681d6df91

Ben Hutchings <ben@decadent.org.uk>2017-09-29 14:32:09 (GMT)
thp: Enable TRANSPARENT_HUGEPAGE_ALWAYS instead of TRANSPARENT_HUGEPAGE_MADVISE
As advised by Andrea Arcangeli - since commit 444eb2a449ef "mm: thp:
set THP defrag by default to madvise and add a stall-free defrag
option" this will generally be best for performance.

So maybe we should weaken the language against THP. Maybe present the
known facts so far, even if the post 4.6 situation is vague/unknown:
before Linux 4.6 there were repeated reports of THP problems with
Postgres, Linux >= 4.6 might improve things but this isn't confirmed.
And it would be good if somebody could run benchmarks on pre 4.6 and
post 4.6 kernels. I would love to but have no access to big (or
medium) hardware.


pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] taking stdbool.h into use
Next
From: Thomas Munro
Date:
Subject: Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)