[GENERAL] Re: shared memory - Mailing list pgsql-general

I have further researched this topic to find out that in the shmget man page
it says:

     [EINVAL]      Size specified is greater than the size of the previously
                   existing segment.  Size specified is less than the system
                   imposed minimum, or greater than the system imposed maxi-
                   mum.

This is the error you get.

However, what you get is not related to BSD/OS shared memory limits. My
'limit' errors were resolved by increased the BSD/OS kernel options as
described. I unsed to get the error:

IpcMemoryCreate: shmget failed (Cannot allocate memory) key=5432101,
size=3070976, permission=600

Note the diffrerent errors (in the parenthesis). Actually, the BSD/OS 3.x
shmget man page does not specify that ENOMEM (my problem) can be returned. :-)

Hope this helps find where this problem is - in my opinion PostgreSQL wrongly
makes assumptions about shared memory allocation, that might happen to work on
some platforms (e.g. Linux), but perhaps not on others.

Regards,
Daniel

>>>"John Henderson" said:
 > Hi y'all,
 > Here is the latest update.
 >
 > The problem is with PostgreSQL allocating shared memory in BSD/OS 3.0.
 > The system has 128M of RAM and 262M of swap.
 > Currently postgres starts as:
 >
 > su -l postgres -c '/usr/local/pgsql/bin/postmaster -i -d 3 -B 887 -o "-eF
 > -S 1024" >> /var/log/postgres 2>&1 &'
 >
 > without problems. However, try to start with -B 1024 (which allocates 1024
 > of 8K blocks used for shared memory buffers)
 > and the following is logged:
 >
 > FindExec: found "/usr/local/pgsql/bin/postgres" using argv[0]
 > binding ShmemCreate(key=52e2c1, size=8852184)
 > IpcMemoryCreate: shmget failed (Invalid argument) key=5432001, size=8852184,
 > per
 > mission=600
 > FATAL 1:  ShmemCreate: cannot create region
 > proc_exit(0) [#0]
 > shmem_exit(0) [#0]
 > exit(0)
 >
 > OK, so we have to get BSD/OS to allocate more shared memory.
 >
 > Note that the actual postgres error is related to calloc and malloc and
 > something else that specifically tells me to increase -B.
 > When -B is increased to 1024 the daemon dies as soon as it starts. The error
 > is not with memory usage but with memory allocation.
 >
 > In this case setting
 >
 > #options                        SYSVSHM
 > #options                        SYSVSEM
 > #options                        SYSVMSG
 >
 > have no value. In some older version of BSD these would tell the compiler to
 > include the optional shared memory header files which are automatically
 > included in this version of BSD so these configs are redundant.
 >
 > In previous attempts I have mucked with:
 > #options        "KMEMSIZE=\(16*1024*1024\)"
 > #options        "DFLDSIZ=\(32*1024*1024\)"
 > #options        "DFLSSIZ=\(4*1024*1024\)"
 > #options        "SOMAXCONN=128"
 > #options                "MAXDSIZ=\(256*1024*1024\)"
 > and made them available to the shell with ulimit. All to no avail.
 > Currently I use the default values from BSD so my front end processes get:
 > moe:~ $ ulimit -a
 > core file size (blocks)  unlimited
 > data seg size (kbytes)   32768
 > file size (blocks)       unlimited
 > max memory size (kbytes) 126280
 > stack size (kbytes)      32768
 > cpu time (seconds)       unlimited
 > max user processes       64
 > pipe size (512 bytes)    2
 > open files               128
 > virtual memory (kbytes)  65536
 >
 > The backend of PostgreSQL is booted from a script /etc/rc.postgres with
 > ulimit -d unlimited just before the daemon is started so I am assuming that
 > postgres gets lots of data space.
 > (In fact I have checked this out and it is true).
 >
 > Adjusting
 > #options                                "SHMMAXPGS=4096"
 > has no effect. The BSD default seems to be 2048 pages and pages are 4K.
 > That means the BSD default is around 8M which is just above my -B 886 (*8 =
 > 7.5M) allocation for shared memory.
 > Allowing the system to soak up some shared mem for whatever reason I would
 > say this is the variable that I have to adjust.
 > So why doesn't it adjust!!??
 > Sin'ichiro M from sdi-users suggested it might be related to semaphore
 > identifiers:
 > but when I used a config from Daniel Kalchev
 > options         "KMAPENTRIES=4000" #prevents kmem malloc errors
 > options                 "SHMMAXPGS=32768"
 > options                 "SHMMNI=400"
 > options                 "SHMSEG=204"
 > options                 "SEMMNS=600"
 > Nothing different happens.
 >
 > Finally there is a tip from Bruce that I need to increase SYSPTSIZE, the
 > number of dynamically allocated page tables for the kernel.
 > SYSPTSIZE is discovered by
 > bpatch -r sysptsize
 > In my case 28.
 > You can adjust this directly into the finished kernel by
 > bpatch sysptsize 40
 > Don't forget to reboot so that the change becomes effective.
 > This should actually allow me to allocate 12*4 = 48 extra Megs of shared
 > memory.
 > Guess what, no difference.
 >
 > Thanks to anyone who has read this far.
 > Here are some questions:
 > 1) What part of memory do shared memory buffers get loaded into: kernel?
 > data? This is to determine which of the ulimitable variables to concentrate
 > on.
 > 2) Is it universally agreed that SHMMAXPGS is the key variable to address
 > and we just need to figure out what is related directly to it?
 > 3) I don't have a clue - do you?
 > Best,
 > John Henderson
 >
 >
 >
 >



************