Re: Out of Memory errors are frustrating as heck! - Mailing list pgsql-performance

From Tom Lane
Subject Re: Out of Memory errors are frustrating as heck!
Date
Msg-id 1095.1555383835@sss.pgh.pa.us
Whole thread Raw
In response to Re: Out of Memory errors are frustrating as heck!  (Gunther <raj@gusw.net>)
List pgsql-performance
Gunther <raj@gusw.net> writes:
> So what I am wondering now, is there seems to be an EXPLOSION of memory 
> consumption near the time of the crash. That ExecutorState has 
> 2,238,648,944 but just until the very last second(s) the RES memory as 
> per top was 1.5 GB I swear.

That's not hugely surprising really, especially in a complex query.
It could be performing some preliminary join that doesn't leak, and
then when it starts to perform the join that does have the leak,
kaboom.  Also, given that you seem to be invoking multi-batch joins,
maybe the preliminary phase is fine and there's only a leak when
reading back a batch.

Anyway, the upshot is that you need to investigate what's happening
while the memory consumption is increasing.  The behavior before
that starts to happen isn't going to be very interesting.  It might
be a bit tricky to catch that if it only takes a few seconds to blow
up, but you could try "c 10000" or so to step through a lot of
AllocSetAlloc calls, repeating till the bad stuff starts to happen,
and then going back to looking at just where the calls are coming
from.

            regards, tom lane



pgsql-performance by date:

Previous
From: Tom Lane
Date:
Subject: Re: Out of Memory errors are frustrating as heck!
Next
From: Gunther
Date:
Subject: Re: Out of Memory errors are frustrating as heck!