Thread: shared memory
Hi y'all, Here is the latest update. The problem is with PostgreSQL allocating shared memory in BSD/OS 3.0. The system has 128M of RAM and 262M of swap. Currently postgres starts as: su -l postgres -c '/usr/local/pgsql/bin/postmaster -i -d 3 -B 887 -o "-eF -S 1024" >> /var/log/postgres 2>&1 &' without problems. However, try to start with -B 1024 (which allocates 1024 of 8K blocks used for shared memory buffers) and the following is logged: FindExec: found "/usr/local/pgsql/bin/postgres" using argv[0] binding ShmemCreate(key=52e2c1, size=8852184) IpcMemoryCreate: shmget failed (Invalid argument) key=5432001, size=8852184, per mission=600 FATAL 1: ShmemCreate: cannot create region proc_exit(0) [#0] shmem_exit(0) [#0] exit(0) OK, so we have to get BSD/OS to allocate more shared memory. Note that the actual postgres error is related to calloc and malloc and something else that specifically tells me to increase -B. When -B is increased to 1024 the daemon dies as soon as it starts. The error is not with memory usage but with memory allocation. In this case setting #options SYSVSHM #options SYSVSEM #options SYSVMSG have no value. In some older version of BSD these would tell the compiler to include the optional shared memory header files which are automatically included in this version of BSD so these configs are redundant. In previous attempts I have mucked with: #options "KMEMSIZE=\(16*1024*1024\)" #options "DFLDSIZ=\(32*1024*1024\)" #options "DFLSSIZ=\(4*1024*1024\)" #options "SOMAXCONN=128" #options "MAXDSIZ=\(256*1024*1024\)" and made them available to the shell with ulimit. All to no avail. Currently I use the default values from BSD so my front end processes get: moe:~ $ ulimit -a core file size (blocks) unlimited data seg size (kbytes) 32768 file size (blocks) unlimited max memory size (kbytes) 126280 stack size (kbytes) 32768 cpu time (seconds) unlimited max user processes 64 pipe size (512 bytes) 2 open files 128 virtual memory (kbytes) 65536 The backend of PostgreSQL is booted from a script /etc/rc.postgres with ulimit -d unlimited just before the daemon is started so I am assuming that postgres gets lots of data space. (In fact I have checked this out and it is true). Adjusting #options "SHMMAXPGS=4096" has no effect. The BSD default seems to be 2048 pages and pages are 4K. That means the BSD default is around 8M which is just above my -B 886 (*8 = 7.5M) allocation for shared memory. Allowing the system to soak up some shared mem for whatever reason I would say this is the variable that I have to adjust. So why doesn't it adjust!!?? Sin'ichiro M from sdi-users suggested it might be related to semaphore identifiers: but when I used a config from Daniel Kalchev options "KMAPENTRIES=4000" #prevents kmem malloc errors options "SHMMAXPGS=32768" options "SHMMNI=400" options "SHMSEG=204" options "SEMMNS=600" Nothing different happens. Finally there is a tip from Bruce that I need to increase SYSPTSIZE, the number of dynamically allocated page tables for the kernel. SYSPTSIZE is discovered by bpatch -r sysptsize In my case 28. You can adjust this directly into the finished kernel by bpatch sysptsize 40 Don't forget to reboot so that the change becomes effective. This should actually allow me to allocate 12*4 = 48 extra Megs of shared memory. Guess what, no difference. Thanks to anyone who has read this far. Here are some questions: 1) What part of memory do shared memory buffers get loaded into: kernel? data? This is to determine which of the ulimitable variables to concentrate on. 2) Is it universally agreed that SHMMAXPGS is the key variable to address and we just need to figure out what is related directly to it? 3) I don't have a clue - do you? Best, John Henderson
I have further researched this topic to find out that in the shmget man page
it says:
[EINVAL] Size specified is greater than the size of the previously
existing segment. Size specified is less than the system
imposed minimum, or greater than the system imposed maxi-
mum.
This is the error you get.
However, what you get is not related to BSD/OS shared memory limits. My
'limit' errors were resolved by increased the BSD/OS kernel options as
described. I unsed to get the error:
IpcMemoryCreate: shmget failed (Cannot allocate memory) key=5432101,
size=3070976, permission=600
Note the diffrerent errors (in the parenthesis). Actually, the BSD/OS 3.x
shmget man page does not specify that ENOMEM (my problem) can be returned. :-)
Hope this helps find where this problem is - in my opinion PostgreSQL wrongly
makes assumptions about shared memory allocation, that might happen to work on
some platforms (e.g. Linux), but perhaps not on others.
Regards,
Daniel
>>>"John Henderson" said:
> Hi y'all,
> Here is the latest update.
>
> The problem is with PostgreSQL allocating shared memory in BSD/OS 3.0.
> The system has 128M of RAM and 262M of swap.
> Currently postgres starts as:
>
> su -l postgres -c '/usr/local/pgsql/bin/postmaster -i -d 3 -B 887 -o "-eF
> -S 1024" >> /var/log/postgres 2>&1 &'
>
> without problems. However, try to start with -B 1024 (which allocates 1024
> of 8K blocks used for shared memory buffers)
> and the following is logged:
>
> FindExec: found "/usr/local/pgsql/bin/postgres" using argv[0]
> binding ShmemCreate(key=52e2c1, size=8852184)
> IpcMemoryCreate: shmget failed (Invalid argument) key=5432001, size=8852184,
> per
> mission=600
> FATAL 1: ShmemCreate: cannot create region
> proc_exit(0) [#0]
> shmem_exit(0) [#0]
> exit(0)
>
> OK, so we have to get BSD/OS to allocate more shared memory.
>
> Note that the actual postgres error is related to calloc and malloc and
> something else that specifically tells me to increase -B.
> When -B is increased to 1024 the daemon dies as soon as it starts. The error
> is not with memory usage but with memory allocation.
>
> In this case setting
>
> #options SYSVSHM
> #options SYSVSEM
> #options SYSVMSG
>
> have no value. In some older version of BSD these would tell the compiler to
> include the optional shared memory header files which are automatically
> included in this version of BSD so these configs are redundant.
>
> In previous attempts I have mucked with:
> #options "KMEMSIZE=\(16*1024*1024\)"
> #options "DFLDSIZ=\(32*1024*1024\)"
> #options "DFLSSIZ=\(4*1024*1024\)"
> #options "SOMAXCONN=128"
> #options "MAXDSIZ=\(256*1024*1024\)"
> and made them available to the shell with ulimit. All to no avail.
> Currently I use the default values from BSD so my front end processes get:
> moe:~ $ ulimit -a
> core file size (blocks) unlimited
> data seg size (kbytes) 32768
> file size (blocks) unlimited
> max memory size (kbytes) 126280
> stack size (kbytes) 32768
> cpu time (seconds) unlimited
> max user processes 64
> pipe size (512 bytes) 2
> open files 128
> virtual memory (kbytes) 65536
>
> The backend of PostgreSQL is booted from a script /etc/rc.postgres with
> ulimit -d unlimited just before the daemon is started so I am assuming that
> postgres gets lots of data space.
> (In fact I have checked this out and it is true).
>
> Adjusting
> #options "SHMMAXPGS=4096"
> has no effect. The BSD default seems to be 2048 pages and pages are 4K.
> That means the BSD default is around 8M which is just above my -B 886 (*8 =
> 7.5M) allocation for shared memory.
> Allowing the system to soak up some shared mem for whatever reason I would
> say this is the variable that I have to adjust.
> So why doesn't it adjust!!??
> Sin'ichiro M from sdi-users suggested it might be related to semaphore
> identifiers:
> but when I used a config from Daniel Kalchev
> options "KMAPENTRIES=4000" #prevents kmem malloc errors
> options "SHMMAXPGS=32768"
> options "SHMMNI=400"
> options "SHMSEG=204"
> options "SEMMNS=600"
> Nothing different happens.
>
> Finally there is a tip from Bruce that I need to increase SYSPTSIZE, the
> number of dynamically allocated page tables for the kernel.
> SYSPTSIZE is discovered by
> bpatch -r sysptsize
> In my case 28.
> You can adjust this directly into the finished kernel by
> bpatch sysptsize 40
> Don't forget to reboot so that the change becomes effective.
> This should actually allow me to allocate 12*4 = 48 extra Megs of shared
> memory.
> Guess what, no difference.
>
> Thanks to anyone who has read this far.
> Here are some questions:
> 1) What part of memory do shared memory buffers get loaded into: kernel?
> data? This is to determine which of the ulimitable variables to concentrate
> on.
> 2) Is it universally agreed that SHMMAXPGS is the key variable to address
> and we just need to figure out what is related directly to it?
> 3) I don't have a clue - do you?
> Best,
> John Henderson
>
>
>
>
************
I have further researched this topic to find out that in the shmget man page
it says:
[EINVAL] Size specified is greater than the size of the previously
existing segment. Size specified is less than the system
imposed minimum, or greater than the system imposed maxi-
mum.
This is the error you get.
However, what you get is not related to BSD/OS shared memory limits. My
'limit' errors were resolved by increased the BSD/OS kernel options as
described. I unsed to get the error:
IpcMemoryCreate: shmget failed (Cannot allocate memory) key=5432101,
size=3070976, permission=600
Note the diffrerent errors (in the parenthesis). Actually, the BSD/OS 3.x
shmget man page does not specify that ENOMEM (my problem) can be returned. :-)
Hope this helps find where this problem is - in my opinion PostgreSQL wrongly
makes assumptions about shared memory allocation, that might happen to work on
some platforms (e.g. Linux), but perhaps not on others.
Regards,
Daniel
>>>"John Henderson" said:
> Hi y'all,
> Here is the latest update.
>
> The problem is with PostgreSQL allocating shared memory in BSD/OS 3.0.
> The system has 128M of RAM and 262M of swap.
> Currently postgres starts as:
>
> su -l postgres -c '/usr/local/pgsql/bin/postmaster -i -d 3 -B 887 -o "-eF
> -S 1024" >> /var/log/postgres 2>&1 &'
>
> without problems. However, try to start with -B 1024 (which allocates 1024
> of 8K blocks used for shared memory buffers)
> and the following is logged:
>
> FindExec: found "/usr/local/pgsql/bin/postgres" using argv[0]
> binding ShmemCreate(key=52e2c1, size=8852184)
> IpcMemoryCreate: shmget failed (Invalid argument) key=5432001, size=8852184,
> per
> mission=600
> FATAL 1: ShmemCreate: cannot create region
> proc_exit(0) [#0]
> shmem_exit(0) [#0]
> exit(0)
>
> OK, so we have to get BSD/OS to allocate more shared memory.
>
> Note that the actual postgres error is related to calloc and malloc and
> something else that specifically tells me to increase -B.
> When -B is increased to 1024 the daemon dies as soon as it starts. The error
> is not with memory usage but with memory allocation.
>
> In this case setting
>
> #options SYSVSHM
> #options SYSVSEM
> #options SYSVMSG
>
> have no value. In some older version of BSD these would tell the compiler to
> include the optional shared memory header files which are automatically
> included in this version of BSD so these configs are redundant.
>
> In previous attempts I have mucked with:
> #options "KMEMSIZE=\(16*1024*1024\)"
> #options "DFLDSIZ=\(32*1024*1024\)"
> #options "DFLSSIZ=\(4*1024*1024\)"
> #options "SOMAXCONN=128"
> #options "MAXDSIZ=\(256*1024*1024\)"
> and made them available to the shell with ulimit. All to no avail.
> Currently I use the default values from BSD so my front end processes get:
> moe:~ $ ulimit -a
> core file size (blocks) unlimited
> data seg size (kbytes) 32768
> file size (blocks) unlimited
> max memory size (kbytes) 126280
> stack size (kbytes) 32768
> cpu time (seconds) unlimited
> max user processes 64
> pipe size (512 bytes) 2
> open files 128
> virtual memory (kbytes) 65536
>
> The backend of PostgreSQL is booted from a script /etc/rc.postgres with
> ulimit -d unlimited just before the daemon is started so I am assuming that
> postgres gets lots of data space.
> (In fact I have checked this out and it is true).
>
> Adjusting
> #options "SHMMAXPGS=4096"
> has no effect. The BSD default seems to be 2048 pages and pages are 4K.
> That means the BSD default is around 8M which is just above my -B 886 (*8 =
> 7.5M) allocation for shared memory.
> Allowing the system to soak up some shared mem for whatever reason I would
> say this is the variable that I have to adjust.
> So why doesn't it adjust!!??
> Sin'ichiro M from sdi-users suggested it might be related to semaphore
> identifiers:
> but when I used a config from Daniel Kalchev
> options "KMAPENTRIES=4000" #prevents kmem malloc errors
> options "SHMMAXPGS=32768"
> options "SHMMNI=400"
> options "SHMSEG=204"
> options "SEMMNS=600"
> Nothing different happens.
>
> Finally there is a tip from Bruce that I need to increase SYSPTSIZE, the
> number of dynamically allocated page tables for the kernel.
> SYSPTSIZE is discovered by
> bpatch -r sysptsize
> In my case 28.
> You can adjust this directly into the finished kernel by
> bpatch sysptsize 40
> Don't forget to reboot so that the change becomes effective.
> This should actually allow me to allocate 12*4 = 48 extra Megs of shared
> memory.
> Guess what, no difference.
>
> Thanks to anyone who has read this far.
> Here are some questions:
> 1) What part of memory do shared memory buffers get loaded into: kernel?
> data? This is to determine which of the ulimitable variables to concentrate
> on.
> 2) Is it universally agreed that SHMMAXPGS is the key variable to address
> and we just need to figure out what is related directly to it?
> 3) I don't have a clue - do you?
> Best,
> John Henderson
>
>
>
>
************
Was this ever resolved on BSDI and PostgreSQL? [ Charset ISO-8859-1 unsupported, converting... ] > Hi y'all, > Here is the latest update. > > The problem is with PostgreSQL allocating shared memory in BSD/OS 3.0. > The system has 128M of RAM and 262M of swap. > Currently postgres starts as: > > su -l postgres -c '/usr/local/pgsql/bin/postmaster -i -d 3 -B 887 -o "-eF > -S 1024" >> /var/log/postgres 2>&1 &' > > without problems. However, try to start with -B 1024 (which allocates 1024 > of 8K blocks used for shared memory buffers) > and the following is logged: > > FindExec: found "/usr/local/pgsql/bin/postgres" using argv[0] > binding ShmemCreate(key=52e2c1, size=8852184) > IpcMemoryCreate: shmget failed (Invalid argument) key=5432001, size=8852184, > per > mission=600 > FATAL 1: ShmemCreate: cannot create region > proc_exit(0) [#0] > shmem_exit(0) [#0] > exit(0) > > OK, so we have to get BSD/OS to allocate more shared memory. > > Note that the actual postgres error is related to calloc and malloc and > something else that specifically tells me to increase -B. > When -B is increased to 1024 the daemon dies as soon as it starts. The error > is not with memory usage but with memory allocation. > > In this case setting > > #options SYSVSHM > #options SYSVSEM > #options SYSVMSG > > have no value. In some older version of BSD these would tell the compiler to > include the optional shared memory header files which are automatically > included in this version of BSD so these configs are redundant. > > In previous attempts I have mucked with: > #options "KMEMSIZE=\(16*1024*1024\)" > #options "DFLDSIZ=\(32*1024*1024\)" > #options "DFLSSIZ=\(4*1024*1024\)" > #options "SOMAXCONN=128" > #options "MAXDSIZ=\(256*1024*1024\)" > and made them available to the shell with ulimit. All to no avail. > Currently I use the default values from BSD so my front end processes get: > moe:~ $ ulimit -a > core file size (blocks) unlimited > data seg size (kbytes) 32768 > file size (blocks) unlimited > max memory size (kbytes) 126280 > stack size (kbytes) 32768 > cpu time (seconds) unlimited > max user processes 64 > pipe size (512 bytes) 2 > open files 128 > virtual memory (kbytes) 65536 > > The backend of PostgreSQL is booted from a script /etc/rc.postgres with > ulimit -d unlimited just before the daemon is started so I am assuming that > postgres gets lots of data space. > (In fact I have checked this out and it is true). > > Adjusting > #options "SHMMAXPGS=4096" > has no effect. The BSD default seems to be 2048 pages and pages are 4K. > That means the BSD default is around 8M which is just above my -B 886 (*8 = > 7.5M) allocation for shared memory. > Allowing the system to soak up some shared mem for whatever reason I would > say this is the variable that I have to adjust. > So why doesn't it adjust!!?? > Sin'ichiro M from sdi-users suggested it might be related to semaphore > identifiers: > but when I used a config from Daniel Kalchev > options "KMAPENTRIES=4000" #prevents kmem malloc errors > options "SHMMAXPGS=32768" > options "SHMMNI=400" > options "SHMSEG=204" > options "SEMMNS=600" > Nothing different happens. > > Finally there is a tip from Bruce that I need to increase SYSPTSIZE, the > number of dynamically allocated page tables for the kernel. > SYSPTSIZE is discovered by > bpatch -r sysptsize > In my case 28. > You can adjust this directly into the finished kernel by > bpatch sysptsize 40 > Don't forget to reboot so that the change becomes effective. > This should actually allow me to allocate 12*4 = 48 extra Megs of shared > memory. > Guess what, no difference. > > Thanks to anyone who has read this far. > Here are some questions: > 1) What part of memory do shared memory buffers get loaded into: kernel? > data? This is to determine which of the ulimitable variables to concentrate > on. > 2) Is it universally agreed that SHMMAXPGS is the key variable to address > and we just need to figure out what is related directly to it? > 3) I don't have a clue - do you? > Best, > John Henderson > > > > > > ************ > > -- Bruce Momjian | http://www.op.net/~candle pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
OK, John, I think I have an answer for you. It is not pretty. You
actually got pretty close.
First, increase SHMMAXPGS by 1024 for every additional 4MB of shared
memory:
/sys/sys/shm.h:69:#define SHMMAXPGS 1024 /* max hardware pages...
The default setting of 1024 is for a maximum of 4MB of shared memory.
Second, use bpatch to find the sysptsize value for the current kernel.
This is computed dynamically at bootup.
$ bpatch -r sysptsize
0x9 = 9
Next, change SYSPTSIZE to a hard-coded value. Use the bpatch value,
plus add 1 for every additional 4MB of shared memory you desire.
/sys/i386/i386/i386_param.c:28:#define SYSPTSIZE 0 /* dynamically...
sysptsize can not be changed by sysctl on the fly.
This should clearly be easier to do on BSDI. I will add a BSDI FAQ for
PostgreSQL to cover this. One downside is that shared memory is not
pageable. It is locked in RAM.
Sorry I am getting to this so late. I just finished my book.
--
Bruce Momjian | http://www.op.net/~candle
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026