Re: Big trouble with memory !! - Mailing list pgsql-general
From | Richard Huxton |
---|---|
Subject | Re: Big trouble with memory !! |
Date | |
Msg-id | 425412BE.5080709@archonet.com Whole thread Raw |
In response to | Re: Big trouble with memory !! (Hervé Piedvache <herve@elma.fr>) |
List | pgsql-general |
Hervé Piedvache wrote: > On Wednesday 06 April 2005 17:03, Richard Huxton wrote: > >>Hervé Piedvache wrote: >> >>>Hi, >>> >>>We have switched to kernel 2.6.11.6 from kernel 2.4.26 ... since this >>>date we have many troubles with PostgreSQL and most of them seems to be >>>memory troubles. >>> >>>As far as we can see, kernel kills the postmaster process when it begins >>>to use swap. You can see the output from dmesg at the bottom of the >>>message. The first thing I am not sure to understand is that the kernel >>>should kill processes to reallocate memory only when physical memory and >>>swap memory are exhausted, shouldn't it ? >>>Second thing: it seems to be related to our kernel switch as it did not >>>happen before that. >>> >>>This can occur when queries / a vacuum require too much memory to run. >>>I have configured my kernel with these options: >>># shared mem >>>kernel.shmmax= 641604096 >>># semaphore >>>kernel.sem = 250 32000 100 400 >>>fs.file-max=65536 >>># overcommit >>>vm.overcommit_memory=2 >>> >>>Does anyone can explain me why I have this problem and how to resolve it >>>? >>> >>>This server is a dedicated PostgreSQL server with 4Gb of RAM. >> >>You might want to try vm.overcommit_memory=1. You don't appear to be the >>only one suffering from an over-zealous oom-killer. >> > > > But if you look at the documentation of PostgreSQL : > ------------------------------------------------------------------------------------------------- > 16.5.3. Linux Memory Overcommit > > In Linux 2.4 and later, the default virtual memory behavior is not optimal for > PostgreSQL. Because of the way that the kernel implements memory overcommit, > the kernel may terminate the PostgreSQL server (the postmaster process) if > the memory demands of another process cause the system to run out of virtual > memory. > > If this happens, you will see a kernel message that looks like this (consult > your system documentation and configuration on where to look for such a > message): > > Out of Memory: Killed process 12345 (postmaster). > > This indicates that the postmaster process has been terminated due to memory > pressure. Although existing database connections will continue to function > normally, no new connections will be accepted. To recover, PostgreSQL will > need to be restarted. > > One way to avoid this problem is to run PostgreSQL on a machine where you can > be sure that other processes will not run the machine out of memory. > > On Linux 2.6 and later, a better solution is to modify the kernel's behavior > so that it will not "overcommit" memory. This is done by selecting strict > overcommit mode via sysctl: > > sysctl -w vm.overcommit_memory=2 > > or placing an equivalent entry in /etc/sysctl.conf. You may also wish to > modify the related setting vm.overcommit_ratio. For details see the kernel > documentation file Documentation/vm/overcommit-accounting. > > Some vendors' Linux 2.4 kernels are reported to have early versions of the 2.6 > overcommit sysctl. However, setting vm.overcommit_memory to 2 on a kernel > that does not have the relevant code will make things worse not better. It is > recommended that you inspect the actual kernel source code (see the function > vm_enough_memory in the file mm/mmap.c) to verify what is supported in your > copy before you try this in a 2.4 installation. The presence of the > overcommit-accounting documentation file should not be taken as evidence that > the feature is there. If in any doubt, consult a kernel expert or your kernel > vendor. > ------------------------------------------------------------------------------------------------- Ah, but I believe there have been some issues with the Linux kernel recently: http://www.kerneltraffic.org/kernel-traffic/kt20050212_296.html#6 If the oom-killer is running riot on your system that would suggest the bug is still there in some form. -- Richard Huxton Archonet Ltd
pgsql-general by date: