Thread: shared memory settings: SHMMAX and SHMALL
I've spent a whole day trying to figure this out. Every FAQ I read, between Linux, Postgres, and Oracle, just sends me further into confusion, so I ask: If I have 512MB of memory in my system, excluding swap space, what values do I want to set for SHMMAX and SHMALL?
> I've spent a whole day trying to figure this out. > Every FAQ I read, between Linux, Postgres, and Oracle, > just sends me further into confusion, so I ask: > > If I have 512MB of memory in my system, excluding swap > space, > what values do I want to set for SHMMAX and SHMALL? That depeneds on your kernel implemetaion and hardware. I found followings in /usr/include/asm/shmparam.h on my Linux box. #define _SHM_ID_BITS 7 #define SHM_ID_MASK ((1<<_SHM_ID_BITS)-1) #define SHM_IDX_SHIFT (_SHM_ID_BITS) #define _SHM_IDX_BITS 15 #define SHM_IDX_MASK ((1<<_SHM_IDX_BITS)-1) /* * _SHM_ID_BITS + _SHM_IDX_BITS must be <= 24 on the i386 and * SHMMAX <= (PAGE_SIZE << _SHM_IDX_BITS). */ here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for the shared memory on this platform if above comment is correct. -- Tatsuo Ishii
Tatsuo Ishii <t-ishii@sra.co.jp> writes: > > If I have 512MB of memory in my system, excluding swap > > space, > > what values do I want to set for SHMMAX and SHMALL? > > That depeneds on your kernel implemetaion and hardware. I found > followings in /usr/include/asm/shmparam.h on my Linux box. > > here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for > the shared memory on this platform if above comment is correct. You can recompile the kernel with the new defaults or just change them on a live system (kernel 2.2.x or higher) with something like: echo 419430400 >/proc/sys/kernel/shmall echo 419430400 >/proc/sys/kernel/shmmax You pick the numbers, of course. -- matthew rice <matt@starnix.com> starnix inc. tollfree: 1-87-pro-linux thornhill, ontario, canada http://www.starnix.com professional linux services & products
Matthew Rice wrote:
> Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> > > If I have 512MB of memory in my system, excluding swap
> > > space,
> > > what values do I want to set for SHMMAX and SHMALL?
> >
> > That depeneds on your kernel implemetaion and hardware. I found
> > followings in /usr/include/asm/shmparam.h on my Linux box.
> >
> > here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for
> > the shared memory on this platform if above comment is correct.
>
> You can recompile the kernel with the new defaults or just change them on
> a live system (kernel 2.2.x or higher) with something like:
>
> echo 419430400 >/proc/sys/kernel/shmall
> echo 419430400 >/proc/sys/kernel/shmmax
>
> You pick the numbers, of course.
Two questions jump to my mind.
1) You've allocated 419430400 bytes (400MB) to shared memory. If I am
planning on running, e.g., 10 postmasters, wouldn't I want to set SHMMAX to
40MB (400MB divided by 10), since SHMMAX is the max shared memory allotment
per running application?
2) I've read that SHMALL = (SHMMAX/PAGE_SIZE*SHMMNI). I haven't been able
to determine SHMMNI's value on my system. I am inferring that PAGE_SIZE = 1
and SHMMNI = 4096 based simply on other users' comments and my
/usr/include/asm/shmparam.h (reprinted below) which, frankly, I am not very
good at reading. Anyway, if this is the case, isn't it likely that
SHMALL is less than SHMMAX?
/*
* Format of a swap-entry for shared memory pages currently out in
* swap space (see also mm/swap.c).
*
* SWP_TYPE = SHM_SWP_TYPE
* SWP_OFFSET is used as follows:
*
* bits 0..6 : id of shared memory segment page belongs to (SHM_ID)
* bits 7..21: index of page within shared memory segment (SHM_IDX)
* (actually fewer bits get used since SHMMAX is so low)
*/
/*
* Keep _SHM_ID_BITS as low as possible since SHMMNI depends on it and
* there is a static array of size SHMMNI.
*/
#define _SHM_ID_BITS 9
#define SHM_ID_MASK ((1<<_SHM_ID_BITS)-1)
#define SHM_IDX_SHIFT (_SHM_ID_BITS)
#define _SHM_IDX_BITS 15
#define SHM_IDX_MASK ((1<<_SHM_IDX_BITS)-1)
/*
* _SHM_ID_BITS + _SHM_IDX_BITS must be <= 24 on the i386 and
* SHMMAX <= (PAGE_SIZE << _SHM_IDX_BITS).
*/
#define SHMMAX 0x2000000 /* max shared seg size (bytes) */
/* Try not to change the default shipped SHMMAX - people rely on it */
#define SHMMIN 1 /* really PAGE_SIZE */ /* min shared seg size (bytes) */
#define SHMMNI (1<<_SHM_ID_BITS) /* max num of segs system wide */
#define SHMALL /* max shm system wide (pages) */ \
(1<<(_SHM_IDX_BITS+_SHM_ID_BITS))
#define SHMLBA PAGE_SIZE /* attach addr a multiple of this */
#define SHMSEG SHMMNI /* max shared segs per process */
#endif /* _ASMI386_SHMPARAM_H */
My last reply was made when I was exhausted. Sorry. Reconsidering.... According to my /usr/include/asm/shmparam.h, SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15) = 128MB. I still don't see where to find the PAGE_SIZE value, although 4096 seems to be a general default out there. Also according to my server's /usr/include/asm/shmparam.h, SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 << (9+15)) = 16. I wonder why some people are setting SHMALL and SHMMAX to the same value when one is a measurement in pages and the other a measurement in bytes. Also, I thought that SHMMAX should be the max allotment in bytes per application, rather than the max shared by all applications. If I anticipate 10 postmasters, is it dangerous to set SHMMAX so high? Tatsuo Ishii wrote: > > I've spent a whole day trying to figure this out. > > Every FAQ I read, between Linux, Postgres, and Oracle, > > just sends me further into confusion, so I ask: > > > > If I have 512MB of memory in my system, excluding swap > > space, > > what values do I want to set for SHMMAX and SHMALL? > > That depeneds on your kernel implemetaion and hardware. I found > followings in /usr/include/asm/shmparam.h on my Linux box. > > #define _SHM_ID_BITS 7 > #define SHM_ID_MASK ((1<<_SHM_ID_BITS)-1) > > #define SHM_IDX_SHIFT (_SHM_ID_BITS) > #define _SHM_IDX_BITS 15 > #define SHM_IDX_MASK ((1<<_SHM_IDX_BITS)-1) > > /* > * _SHM_ID_BITS + _SHM_IDX_BITS must be <= 24 on the i386 and > * SHMMAX <= (PAGE_SIZE << _SHM_IDX_BITS). > */ > > here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for > the shared memory on this platform if above comment is correct. > -- > Tatsuo Ishii
Gregory Bittar <gbittar@iqa.cc> writes: > According to my /usr/include/asm/shmparam.h, > SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15) > = 128MB. Mine is just: #define SHMMAX 0x2000000 > I still don't see where to find the PAGE_SIZE value, although 4096 seems > to be a general default out there. man getpagesize or matt@sol:/usr/include[106]% egrep 'define PAGE_S(IZE|HIFT)' */* 2>/dev/null asm/page.h:#define PAGE_SHIFT 12 asm/page.h:#define PAGE_SIZE (1UL << PAGE_SHIFT) linux/a.out.h:#define PAGE_SIZE 0x400 sys/user.h:#define PAGE_SHIFT 12 sys/user.h:#define PAGE_SIZE (1UL << PAGE_SHIFT) Although, 'man postmaster' says that it used 8k (or "whatever BLCKSZ is set to in config.h") for it's shared memory buffers. > Also according to my server's /usr/include/asm/shmparam.h, > SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 << > (9+15)) = 16. My 2.2.16 kernel has 15 and 7, BTW. > I wonder why some people are setting SHMALL and SHMMAX to the same value > when one is a measurement in pages and the other a measurement in Pure laziness, I'm sure. > bytes. Also, I thought that SHMMAX should be the max allotment in bytes > per application, rather than the max shared by all applications. If I > anticipate 10 postmasters, is it dangerous to set SHMMAX so high? I'm not certain. I just used those numbers for a testing server (I actually only told postmaster to use ~300MB). Are you saying 10 postmasters or 10 backends? Either way, you can set SHMMAX as high as you like. Just don't tell the postmasters to try and use them all. Actually, I just did this: echo 11111111111 >/proc/sys/kernel/shmmax and got: cat /proc/sys/kernel/shmmax -1773790777 Probably not good, long term :) My kernel also wouldn't let me make SHMALL higher than 16M -- matthew rice <matt@starnix.com> starnix inc. tollfree: 1-87-pro-linux thornhill, ontario, canada http://www.starnix.com professional linux services & products
> My last reply was made when I was exhausted. Sorry. > > Reconsidering.... > > According to my /usr/include/asm/shmparam.h, > SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15) > = 128MB. > > I still don't see where to find the PAGE_SIZE value, although 4096 seems > to be a general default out there. On my Linux box, I found: #define PAGE_SHIFT 12 #define PAGE_SIZE (1UL << PAGE_SHIFT) so PAGE_SIZE must be 4096. > Also according to my server's /usr/include/asm/shmparam.h, > SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 << > (9+15)) = 16. > > I wonder why some people are setting SHMALL and SHMMAX to the same value > when one is a measurement in pages and the other a measurement in > bytes. Me too. Probably SHMALL in /proc/sys/kernel/shmall is treated differently from the one defined in those header files? > Also, I thought that SHMMAX should be the max allotment in bytes > per application, rather than the max shared by all applications. If I > anticipate 10 postmasters, is it dangerous to set SHMMAX so high? No, SHMMAX is for all applications (10 postmasters in your case?). I guess you are using x86/Linux and from your mail I would say you could set SHMMAX to 128MB(= 134217728 bytes). Also please make sure that you have enough file table entries in your kernel. You are going to run many postmasters which involves many backends. In my case I set following in /etc/sysctl.conf. fs.file-max = 16384 This works for recent red hat (or derived) distributions. If this doesn't work for you, then you could set a variable in /proc(file-max). -- Tatsuo Ishii