Re: BUG #16707: Memory leak - Mailing list pgsql-bugs

From Kurt Roeckx
Subject Re: BUG #16707: Memory leak
Date
Msg-id 20201110080913.GD391173@roeckx.be
Whole thread Raw
In response to Re: BUG #16707: Memory leak  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-bugs
On Mon, Nov 09, 2020 at 05:20:37PM -0500, Tom Lane wrote:
> Kurt Roeckx <kurt@roeckx.be> writes:
> > On Mon, Nov 09, 2020 at 04:34:33PM -0500, Tom Lane wrote:
> >> * look in postmaster log for memory dump
> 
> > ...
> > Grand total: 3575000 bytes in 533 blocks; 596232 free (450 chunks); 2978768 used
> 
> > Which was for this process:
> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > postgres 10000  2.6 16.3 5547172 5374656 ?     Ss   Nov08  54:10 postgres: synapse synapse [local] idle
> 
> Hm.  It would seem that whatever you're leaking was not allocated via
> palloc.  Have you got any extensions loaded into that backend?

synapse=> \dx
                 List of installed extensions
  Name   | Version |   Schema   |         Description
---------+---------+------------+------------------------------
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
(1 row)

> It's also worth noting that if you've got 4GB of shared buffers,
> a total process vsize of 5.3GB doesn't seem all that far out of
> line.  I'm not quite convinced that you have a leak at all,
> as opposed to processes gradually touching more and more of the
> shared buffer arena.

Top says the shared size is now 4.1 GB, and has said so for a
while. On the other hand, the virtual size keeps growning. The
virtual size shouldn't change much over time. The resident size
will grow until the shared buffer is fully used, then should stay
around that value.

ps now says:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
postgres 10000  2.2 16.6 5632572 5468232 ?     Ss   Nov08  61:09 postgres: synapse synapse [local] idle

At some point all 32 GB of RAM will get used, it starts to swap
out, then malloc starts to fail. (I've set Linux to not
allow overcommit (vm.overcommit_memory = 2), otherwise it would be
the OOM killer doing it's thing.

This is what an other process looks like:
postgres  8966  0.2  3.8 4387236 1276104 ?     Ss   Nov08   7:10 postgres: kurt certs [local] idle

top says that that process has 1.2 GB shared RAM, which also
matches it resident size.


Kurt




pgsql-bugs by date:

Previous
From: Andres Freund
Date:
Subject: Re: BUG #16696: Backend crash in llvmjit
Next
From: Kurt Roeckx
Date:
Subject: Re: BUG #16707: Memory leak