Re: PG Killed by OOM Condition - Mailing list pgsql-hackers

On Mon, Oct 03, 2005 at 01:25:00PM -0700, Jeff Davis wrote:
> It's happened to me...
> Usually it's when there's some other runaway process, and the kernel
> decides to kill PostgreSQL because it can't tell the difference.
> I really don't like that "feature" in linux. Nobody has been able to
> explain to me why linux is the only OS with an OOM Killer. If someone
> here has more information, I'd like to know.
> When using linux I always set vm_overcommit=2.

I don't think it's the only one. Perhaps the only one with a default
setting of on?

I believe Solaris can be configured to over-commit memory.

The problem really comes down to the definition of fork(). UNIX fork()
requires that the system splits a process into two separate copies.
For an application that is currently using 500 Mbytes of virtual
memory, this would require that the system accept that each process,
may use its own complete copy of this 500 Mbytes, for a total of 1
Gbyte in active use. fork() a few more times, and we hit 2 Gbytes, 4
Gbytes - whatever. Even if only for an instant, and even if the pages
are copy-on-write, the system has to consider the possibility that
each application may choose to modify all pages, resulting in complete
copies.

Solaris defaults to not over committing. This means that such an
application, as defined above, would fail at one of the invocations
of fork(). Even though the memory isn't being used - Solaris, by default,
isn't willing to 'over commit' to having the memory allocated as a result
of fork(). Some very large applications don't work under Solaris as a
result, unless this setting is disabled.

Linux takes the opposite extreme. It assumes that copy-on-write will
get us through. The fork() would be allowed - but if available virtual
memory actually does become low, it's faced with a hard decision. It
either fails an application of its choice in a controlled "OOM"
manner, by trying to guess which application is misbehaving, and
deciding to kill that one - or it waits until memory really is gone,
at which point MANY applications may start to fail, as their page
fault fails to allocate a page, and the process dies a horrible death.

It's not an easy decision. Linux isn't wrong. Solaris isn't wrong.
Most people never hit these problems, and the people that do, are
just as likely to hit one problem, or the other. The grass is always
greener on the side of the fence that isn't hurting me right now,
and all that.

Cheers,
mark

-- 
mark@mielke.cc / markm@ncf.ca / markm@nortel.com     __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada
 One ring to rule them all, one ring to find them, one ring to bring them all                      and in the darkness
bindthem...
 
                          http://mark.mielke.cc/