Thread: Huge pages not working on PG 9.4
I've compiled and installed PG 9.4.0 on my Linux VM, but attempts to enable huge pages (huge_pages=on in postgresql.conf) have resulted in this error message in the log: FATAL: huge pages not supported on this platform I'm running CentOS 6.4. Here's the uname output: Linux 2.6.32-358.2.1.el6.x86_64 #1 SMP Wed Mar 13 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux I've set a reasonable nr_hugepages value in /etc/sysctl.conf. Is there some kernel mod or environment variable that I also need? Michael Heaney JCVI
On Jan 16, 2015, at 3:07 PM, Michael Heaney <mheaney@jcvi.org> wrote: > > I've set a reasonable nr_hugepages value in /etc/sysctl.conf. Is there some kernel mod or environment variable that Ialso need? This? <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/sect-Oracle_9i_and_10g_Tuning_Guide-Large_Memory_Optimization_Big_Pages_and_Huge_Pages-Configuring_Huge_Pages_in_Red_Hat_Enterprise_Linux_4_or_5.html> -- Scott Ribe scott_ribe@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice
On 1/16/2015 5:13 PM, Scott Ribe wrote: > On Jan 16, 2015, at 3:07 PM, Michael Heaney <mheaney@jcvi.org> wrote: >> I've set a reasonable nr_hugepages value in /etc/sysctl.conf. Is there some kernel mod or environment variable that Ialso need? > This? > > <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/sect-Oracle_9i_and_10g_Tuning_Guide-Large_Memory_Optimization_Big_Pages_and_Huge_Pages-Configuring_Huge_Pages_in_Red_Hat_Enterprise_Linux_4_or_5.html> > I'll have the SA's add the 'memlock' parameter to /etc/security/limits.conf, and try it again. Thanks! -- Michael Heaney JCVI
On 1/16/2015 5:34 PM, Michael Heaney wrote: > ... > > I'll have the SA's add the 'memlock' parameter to > /etc/security/limits.conf, > and try it again. Thanks! > > - <sigh> Nope, that didn't make any difference. Since I compile my own binary, I'm wondering if there's something that 'configure' didn't set properly. -- Michael Heaney JCVI
On Jan 16, 2015, at 3:50 PM, Michael Heaney <mheaney@jcvi.org> wrote: > > Nope, that didn't make any difference. Since I compile my own binary, > I'm wondering if there's something that 'configure' didn't set properly. Possibly configure needed to see the settings you just made? (I really have idea--that first link I posted exhausted my knowledgeon that subject.) -- Scott Ribe scott_ribe@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice
Michael Heaney <mheaney@jcvi.org> writes:
>> FATAL: huge pages not supported on this platform
> <sigh> Nope, that didn't make any difference. Since I compile my own
> binary,
> I'm wondering if there's something that 'configure' didn't set properly.
A quick look at the PG source code says that that message means that
MAP_HUGETLB isn't defined. Our code is expecting <sys/mman.h> to provide
that symbol, which it should do by default according to what I can see on
my RHEL 6.6 box. Perhaps you've defined some of the feature restriction
macros like _POSIX_SOURCE?
regards, tom lane
On 1/16/2015 6:58 PM, Tom Lane wrote: > Michael Heaney <mheaney@jcvi.org> writes: >>> FATAL: huge pages not supported on this platform >> <sigh> Nope, that didn't make any difference. Since I compile my own >> binary, >> I'm wondering if there's something that 'configure' didn't set properly. > A quick look at the PG source code says that that message means that > MAP_HUGETLB isn't defined. Our code is expecting <sys/mman.h> to provide > that symbol, which it should do by default according to what I can see on > my RHEL 6.6 box. Perhaps you've defined some of the feature restriction > macros like _POSIX_SOURCE? > > regards, tom lane Thanks for the helpful clue, Tom. I haven't knowingly defined _POSIX_SOURCE or any other macros. I go through the same configure/make process on the Centos 6.6 VM on my home PC, and huge pages works just fine. Maybe it's an issue with CentOS 6.4? -- Michael Heaney JCVI
On 1/16/2015 6:39 PM, Scott Ribe wrote: > On Jan 16, 2015, at 3:50 PM, Michael Heaney <mheaney@jcvi.org> wrote: >> Nope, that didn't make any difference. Since I compile my own binary, >> I'm wondering if there's something that 'configure' didn't set properly. > Possibly configure needed to see the settings you just made? (I really have idea--that first link I posted exhausted myknowledge on that subject.) > I re-ran configure/make after rebooting the box, with no difference. Thanks for responding, though. I recall reading that same link a couple years ago, when I was trying to get huge pages to work for Sybase ASE. -- Michael Heaney JCVI
Michael Heaney <mheaney@jcvi.org> writes:
> On 1/16/2015 6:58 PM, Tom Lane wrote:
>> A quick look at the PG source code says that that message means that
>> MAP_HUGETLB isn't defined. Our code is expecting <sys/mman.h> to provide
>> that symbol, which it should do by default according to what I can see on
>> my RHEL 6.6 box. Perhaps you've defined some of the feature restriction
>> macros like _POSIX_SOURCE?
> Thanks for the helpful clue, Tom. I haven't knowingly defined
> _POSIX_SOURCE or any other macros. I go through the same configure/make
> process on the Centos 6.6 VM on my home PC, and huge pages works just
> fine. Maybe it's an issue with CentOS 6.4?
Dunno, hard to believe that hugetlb support wasn't there in 6.4. A quick
troll through the RHEL 6.5 and 6.6 release notes finds something about
better hugetlb support for System Z processors, but what we're discussing
here shouldn't be architecture-specific.
You might try comparing /usr/include/sys/mman.h, /usr/include/bits/mman.h,
and /usr/include/features.h between your 6.4 and 6.6 systems to see if
you can spot any relevant differences.
regards, tom lane
On 1/16/2015 10:52 PM, Tom Lane wrote: > Michael Heaney <mheaney@jcvi.org> writes: >> On 1/16/2015 6:58 PM, Tom Lane wrote: >>> A quick look at the PG source code says that that message means that >>> MAP_HUGETLB isn't defined. Our code is expecting <sys/mman.h> to provide >>> that symbol, which it should do by default according to what I can see on >>> my RHEL 6.6 box. Perhaps you've defined some of the feature restriction >>> macros like _POSIX_SOURCE? >> Thanks for the helpful clue, Tom. I haven't knowingly defined >> _POSIX_SOURCE or any other macros. I go through the same configure/make >> process on the Centos 6.6 VM on my home PC, and huge pages works just >> fine. Maybe it's an issue with CentOS 6.4? > Dunno, hard to believe that hugetlb support wasn't there in 6.4. A quick > troll through the RHEL 6.5 and 6.6 release notes finds something about > better hugetlb support for System Z processors, but what we're discussing > here shouldn't be architecture-specific. > > You might try comparing /usr/include/sys/mman.h, /usr/include/bits/mman.h, > and /usr/include/features.h between your 6.4 and 6.6 systems to see if > you can spot any relevant differences. > > regards, tom lane Nice catch! The following line was missing from /usr/include/bits/mman.h on CentOS 6.4: # define MAP_HUGETLB 0x40000 /* Create huge page mapping. */ I added it to a test 6.4 box, and the rebuilt Postgres was then able to use huge pages. I'm leery about adding this to all the 6.4 boxes, though, without some reassurance that it won't break something. I'll bring it up with my Linux SA's next week. Thanks again for your help! -- Michael Heaney JCVI
> From: Scott Ribe <scott_ribe@elevated-dev.com> > To: Michael Heaney <mheaney@jcvi.org> > Cc: "pgsql-admin@postgresql.org" <pgsql-admin@postgresql.org> > Sent: Friday, 16 January 2015, 23:39 > Subject: Re: [ADMIN] Huge pages not working on PG 9.4 > > On Jan 16, 2015, at 3:50 PM, Michael Heaney <mheaney@jcvi.org> wrote: >> >> Nope, that didn't make any difference. Since I compile my own binary, >> I'm wondering if there's something that 'configure' > didn't set properly. > > Possibly configure needed to see the settings you just made? (I really have > idea--that first link I posted exhausted my knowledge on that subject.) > I've very thin knowledge on this too, but I'll chip in with what I remember. Not sure if you'd have to set hugetlb_shm_groupto the gid of the postgres user. And I think you can see what support your kernel was compiled with this: $ grep HUGE /boot/config-$(uname -r) And a quick peek at what you've got set up currently: $ grep -R "" /sys/kernel/mm/hugepages/ /proc/sys/vm/*huge* $ cat /proc/meminfo |grep Huge Perhaps if you post those to the list it'd help someone advise?