Thread: Setting kernel.shmmax

Setting kernel.shmmax

From
fatih ozturk
Date:
Hi

My question is about shmmax kernel parameter and its importance in tuning postgresql database.

Our database version:

postgres=# select version();
                                                 version
----------------------------------------------------------------------------------------------------------
 PostgreSQL 8.3.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
(1 row)

"uname -a" output:

Linux vt02.deneme.net 2.6.9-42.0.3.ELsmp #1 SMP Fri Oct 6 06:28:26 CDT 2006 x86_64 x86_64 x86_64 GNU/Linux


And some of our kernel parameters are

kernel.shmmni = 4096
kernel.shmall = 3279547
kernel.shmmax = 2147483648

"free"  output

             total       used       free     shared    buffers     cached
Mem:      16415276   14906444    1508832          0     139192   13743608
-/+ buffers/cache:    1023644   15391632
Swap:      4192924        208    4192716

As you see we have 16 GB  RAM and we use about 14 GB of it.
But kernel.shmmax parameter 2 GB.

In most of tuning guides setting shmmax parameter to %90 of total memory is advised.
But even though we didn't do such shmmax tuning "free" command shows that we are using 16 GB of RAM effectively.

How postgresql using RAM effectively without tuned shmmax?
Should we tune shmmax in our stuation?
If so, what is shmmax's proper value for our stuation?

Thanks for your interest


Re: Setting kernel.shmmax

From
Iñigo Martinez Lasala
Date:
Free memory is going to be used by operating system as cache memory. If operating system does not need to use this memory, it will remain as free.
How big is your database? If it is smaller that your RAM you will always get free memory since it won't be used by operating system in order to cache database files.
And, although your database is bigger, only frequently accessed database files will be in cache.

You only need to modify kernel.shmmax in order to accommodate postgres process only, and memory usage by postgres process is mainly controlled by shared_buffers, workmem, etc parameters from postgresql.conf. Remaining memory will be used by operating system as cache memory.


-----Original Message-----
From: fatih ozturk <ozturkfa@yahoo.com>
To: pgsql-admin@postgresql.org
Subject: [ADMIN] Setting kernel.shmmax
Date: Fri, 10 Jul 2009 00:57:09 -0700 (PDT)

Hi

My question is about shmmax kernel parameter and its importance in tuning postgresql database.

Our database version:

postgres=# select version();
                                                 version
----------------------------------------------------------------------------------------------------------
 PostgreSQL 8.3.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
(1 row)

"uname -a" output:

Linux vt02.deneme.net 2.6.9-42.0.3.ELsmp #1 SMP Fri Oct 6 06:28:26 CDT 2006 x86_64 x86_64 x86_64 GNU/Linux


And some of our kernel parameters are

kernel.shmmni = 4096
kernel.shmall = 3279547
kernel.shmmax = 2147483648

"free"  output

             total       used       free     shared    buffers     cached
Mem:      16415276   14906444    1508832          0     139192   13743608
-/+ buffers/cache:    1023644   15391632
Swap:      4192924        208    4192716

As you see we have 16 GB  RAM and we use about 14 GB of it.
But kernel.shmmax parameter 2 GB.

In most of tuning guides setting shmmax parameter to %90 of total memory is advised.
But even though we didn't do such shmmax tuning "free" command shows that we are using 16 GB of RAM effectively.

How postgresql using RAM effectively without tuned shmmax?
Should we tune shmmax in our stuation?
If so, what is shmmax's proper value for our stuation?

Thanks for your interest



Re: Setting kernel.shmmax

From
fatih ozturk
Date:
Thanks for your answer Iñigo Martinez Lasala.
So what i understand from your description below
"
You only need to modify kernel.shmmax in order to accommodate postgres process only, and memory usage by postgres process is mainly controlled by shared_buffers, workmem, etc parameters from postgresql.conf.
"
is kernel.shmmax is about max connection to a database. So shmmax does not effects memory usage of querys directly.

Hımmm...


Am i misunderstand something?

Thanks for your interest



--- On Fri, 7/10/09, Iñigo Martinez Lasala <imartinez@vectorsf.com> wrote:

From: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Subject: Re: [ADMIN] Setting kernel.shmmax
To: "fatih ozturk" <ozturkfa@yahoo.com>
Cc: pgsql-admin@postgresql.org
Date: Friday, July 10, 2009, 9:21 AM

Free memory is going to be used by operating system as cache memory. If operating system does not need to use this memory, it will remain as free.
How big is your database? If it is smaller that your RAM you will always get free memory since it won't be used by operating system in order to cache database files.
And, although your database is bigger, only frequently accessed database files will be in cache.

You only need to modify kernel.shmmax in order to accommodate postgres process only, and memory usage by postgres process is mainly controlled by shared_buffers, workmem, etc parameters from postgresql.conf. Remaining memory will be used by operating system as cache memory.


-----Original Message-----
From: fatih ozturk <ozturkfa@yahoo.com>
To: pgsql-admin@postgresql.org
Subject: [ADMIN] Setting kernel.shmmax
Date: Fri, 10 Jul 2009 00:57:09 -0700 (PDT)

Hi

My question is about shmmax kernel parameter and its importance in tuning postgresql database.

Our database version:

postgres=# select version();
                                                 version
----------------------------------------------------------------------------------------------------------
 PostgreSQL 8.3.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)
(1 row)

"uname -a" output:

Linux vt02.deneme.net 2.6.9-42.0.3.ELsmp #1 SMP Fri Oct 6 06:28:26 CDT 2006 x86_64 x86_64 x86_64 GNU/Linux


And some of our kernel parameters are

kernel.shmmni = 4096
kernel.shmall = 3279547
kernel.shmmax = 2147483648

"free"  output

             total       used       free     shared    buffers     cached
Mem:      16415276   14906444    1508832          0     139192   13743608
-/+ buffers/cache:    1023644   15391632
Swap:      4192924        208    4192716

As you see we have 16 GB  RAM and we use about 14 GB of it.
But kernel.shmmax parameter 2 GB.

In most of tuning guides setting shmmax parameter to %90 of total memory is advised.
But even though we didn't do such shmmax tuning "free" command shows that we are using 16 GB of RAM effectively.

How postgresql using RAM effectively without tuned shmmax?
Should we tune shmmax in our stuation?
If so, what is shmmax's proper value for our stuation?

Thanks for your interest




Re: Setting kernel.shmmax

From
Mike Ivanov
Date:
fatih ozturk wrote:

 > In most of tuning guides setting shmmax
 > parameter to %90 of total memory is advised.

Does not seem a good idea to me.

 > is kernel.shmmax is about max connection to a database.
 > So shmmax does not effects memory usage of querys directly.

No.

OK, there are two parameters. One is called shared_buffers in
posrgesql.conf. This parameter defines the size of a memory block used
by Postgres for buffers, cache and such. This memory region is called
Shared Memory (well, because it is shared between Postgres processes).

The operating system puts certain limits on how big those shared memory
regions could be. The maximum size is defined by kernel.shmmax parameter.

First you decide on how much memory Postgres should use. How much
exactly--it depends. In most cases it is safe to give 1/4 to 1/2 of all
the memory to Postgres, provided there are no other services (like
Apache) running on that machine. In your particular case I'd stick
closer to 1/4 side. Put that value in postgresql.conf.

Now, make kernel.shmmax slightly bigger than shared_buffers to
accommodate that value.

Cheers,
Mike