Re: [GENERAL] Running out of memory the hard way ... - Mailing list pgsql-general

From Tom Lane
Subject Re: [GENERAL] Running out of memory the hard way ...
Date
Msg-id 1182.1486568664@sss.pgh.pa.us
Whole thread Raw
In response to Re: [GENERAL] Running out of memory the hard way ...  (Albe Laurenz <laurenz.albe@wien.gv.at>)
Responses Re: [GENERAL] Running out of memory the hard way ...  (Paul Ramsey <pramsey@cleverelephant.ca>)
Re: [GENERAL] Running out of memory the hard way ...  (Bill Moran <wmoran@potentialtech.com>)
List pgsql-general
Albe Laurenz <laurenz.albe@wien.gv.at> writes:
> Bill Moran wrote:
>> What I feel is the best way to mitigate the situation, is to have some
>> setting that limits the maximum RAM any backend can consume.

> I'd delegate that problem to the operating system which, after all,
> should know best of all how much memory a process uses.

I've had some success using ulimit in the past, although it does have
the disadvantage that you have to impose the same limit on every PG
process.  (You set it before starting the postmaster and it inherits
to every child process.)  If memory serves, limiting with the -v switch
works better than -d or -m on Linux; but I might be misremembering.
Conceivably we could add code to let the ulimit be set per-process,
if the use-case were strong enough.

To implement a limit inside PG, we'd have to add expensive bookkeeping
to the palloc/pfree mechanism, and even that would be no panacea because
it would fail to account for memory allocated directly from malloc.
Hence, you could be pretty certain that it would be wildly inaccurate
for sessions using third-party code such as PostGIS or Python.  An
OS-enforced limit definitely sounds better from here.

            regards, tom lane


pgsql-general by date:

Previous
From: Nikolai Zhubr
Date:
Subject: Re: [GENERAL] Causeless CPU load waves in backend, on windows, 9.5.5(EDB binary).
Next
From: Paul Ramsey
Date:
Subject: Re: [GENERAL] Running out of memory the hard way ...