Re: Big trouble with memory !! - Mailing list pgsql-general

From Martijn van Oosterhout
Subject Re: Big trouble with memory !!
Date
Msg-id 20050406185744.GE14589@svana.org
Whole thread Raw
In response to Re: Big trouble with memory !!  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Big trouble with memory !!  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
On Wed, Apr 06, 2005 at 12:52:55PM -0400, Tom Lane wrote:
> Martijn van Oosterhout <kleptog@svana.org> writes:
> > What I don't understand is that with true strict overcommit, the kernel
> > should never need to kill your process since there is always in
> > principle enough room.
>
> Indeed.  Are you *sure* you have overcommit turned off?  That should
> disable the OOM killer altogether.  You should probably go read the
> kernel documentation rather than assume Postgres' documentation knows
> what it's talking about ;-)

What I don't understand is the problem with overcommitting. Not
overcommitting is rather wasteful of memory and you'll start seeing
failed allocations long before "free" indicates there's a problem. See
the original poster of this thread whose memory allocation for 24KB was
being rejected while there is still megabytes of "free" memory (not to
mention swap). The memory which is "overcommitted" is being used for
disk cache and buffers in the meantime.

Note the kernel message says "postmaster" was killed by the kernel, yet
that was most probably a child process. Both the postmaster, the stats
collector and any child process will be referred to as "postmaster".
AFAIK no exec() is being done to change the name (see
/proc/<pid>/status).

BTW, according to this page in the archive:
http://archives.postgresql.org/pgsql-patches/2003-11/msg00194.php

           2 - (NEW) strict overcommit. The total address space commit
               for the system is not permitted to exceed swap + a
               configurable percentage (default is 50) of physical RAM.
               Depending on the percentage you use, in most situations
               this means a process will not be killed while accessing
               pages but will receive errors on memory allocation as
               appropriate.

So the origial poster had 4GB of RAM and 2GB of swap, so by that rule
he's only allowed to allocate a total of (swap)2GB+ 50%(memory)4GB =
4GB. In other words, he'd never use swap at all. Maybe fiddling that
percentage would be a better idea. But overcommit still seems useless
to me.

PS, I only know about Linux here.
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Attachment

pgsql-general by date:

Previous
From: "Otto Blomqvist"
Date:
Subject: Problems with Set Returning Functions (SRFs)
Next
From: Mage
Date:
Subject: Re: lower function