Re: Big trouble with memory !! - Mailing list pgsql-general

On Wednesday 06 April 2005 17:03, Richard Huxton wrote:
> Hervé Piedvache wrote:
> > Hi,
> >
> > We have switched to kernel 2.6.11.6 from kernel 2.4.26 ... since this
> > date we have many troubles with PostgreSQL and most of them seems to be
> > memory troubles.
> >
> > As far as we can see, kernel kills the postmaster process when it begins
> > to use swap. You can see the output from dmesg at the bottom of the
> > message. The first thing I am not sure to understand is that the kernel
> > should kill processes to reallocate memory only when physical memory and
> > swap memory are exhausted, shouldn't it ?
> > Second thing: it seems to be related to our kernel switch as it did not
> > happen before that.
> >
> > This can occur when queries / a vacuum require too much memory to run.
> > I have configured my kernel with these options:
> > # shared mem
> > kernel.shmmax= 641604096
> > # semaphore
> > kernel.sem = 250 32000 100 400
> > fs.file-max=65536
> > # overcommit
> > vm.overcommit_memory=2
> >
> > Does anyone can explain me why I have this problem and how to resolve it
> > ?
> >
> > This server is a dedicated PostgreSQL server with 4Gb of RAM.
>
> You might want to try vm.overcommit_memory=1. You don't appear to be the
> only one suffering from an over-zealous oom-killer.
>

But if you look at the documentation of PostgreSQL :
-------------------------------------------------------------------------------------------------
16.5.3. Linux Memory Overcommit

In Linux 2.4 and later, the default virtual memory behavior is not optimal for
PostgreSQL. Because of the way that the kernel implements memory overcommit,
the kernel may terminate the PostgreSQL server (the postmaster process) if
the memory demands of another process cause the system to run out of virtual
memory.

If this happens, you will see a kernel message that looks like this (consult
your system documentation and configuration on where to look for such a
message):

Out of Memory: Killed process 12345 (postmaster).

This indicates that the postmaster process has been terminated due to memory
pressure. Although existing database connections will continue to function
normally, no new connections will be accepted. To recover, PostgreSQL will
need to be restarted.

One way to avoid this problem is to run PostgreSQL on a machine where you can
be sure that other processes will not run the machine out of memory.

On Linux 2.6 and later, a better solution is to modify the kernel's behavior
so that it will not "overcommit" memory. This is done by selecting strict
overcommit mode via sysctl:

sysctl -w vm.overcommit_memory=2

or placing an equivalent entry in /etc/sysctl.conf. You may also wish to
modify the related setting vm.overcommit_ratio. For details see the kernel
documentation file Documentation/vm/overcommit-accounting.

Some vendors' Linux 2.4 kernels are reported to have early versions of the 2.6
overcommit sysctl. However, setting vm.overcommit_memory to 2 on a kernel
that does not have the relevant code will make things worse not better. It is
recommended that you inspect the actual kernel source code (see the function
vm_enough_memory in the file mm/mmap.c) to verify what is supported in your
copy before you try this in a 2.4 installation. The presence of the
overcommit-accounting documentation file should not be taken as evidence that
the feature is there. If in any doubt, consult a kernel expert or your kernel
vendor.
-------------------------------------------------------------------------------------------------

Where is the good solution ?
--
Hervé Piedvache

NOUVELLE ADRESSE - NEW ADDRESS :
Elma Ingénierie Informatique
3 rue d'Uzès
F-75002 - Paris - France
Pho. 33-144949901
Fax. 33-144882747