Re: Understanding Postgres Memory Usage - Mailing list pgsql-general

From Theron Luhn
Subject Re: Understanding Postgres Memory Usage
Date
Msg-id CAHYFdT-BRT2yHQh0qXn_XXfKO4H_NtLT0DPN2-pHNNPwMioHSw@mail.gmail.com
Whole thread Raw
In response to Re: Understanding Postgres Memory Usage  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Understanding Postgres Memory Usage  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
> It would be worth using plain old top to watch this process.  We have
> enough experience with that to be pretty sure how to interpret its
> numbers: "RES minus SHR" is the value to be worried about.


RES - SHR is showing a similar increase to what smem is reporting.

— Theron

On Thu, Aug 25, 2016 at 11:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Theron Luhn <theron@luhn.com> writes:
>> If it's not an outright leak, it's probably consumption of cache space.
>> We cache stuff that we've read from system catalogs, so sessions that
>> touch lots of tables (like thousands) can grow due to that.  Another
>> possible source of large cache consumption is calling lots-and-lots of
>> plpgsql functions.

> I have a reasonable number of tables (around 50) and very few plpgsql
> functions.

Doesn't sound like a lot ...


>> If the same query, repeated over and over, causes memory to continue
>> to grow, I'd call it a leak (ie bug).  If repeat executions consume
>> no additional memory then it's probably intentional caching behavior.

> Here's the results of that:
> https://gist.github.com/luhn/e09522d524354d96d297b153d1479c13

> So kind of a combination of the two:  Memory usage increases up to a
> certain point but then plateaus.  So... cache?  It's ~100MB increase,
> though, which seems an excessive amount.  What could be taking up that much
> cache?

Hmm.  I find it mighty suspicious that the USS, PSS, and RSS numbers are
all increasing to pretty much the same tune, ie from very little to circa
100MB.  I think there is a decent chance that smem is not doing what it
says on the tin, and in fact is including shared memory consumption in
"USS".  In which case the apparent leak just corresponds to the process
gradually touching more and more of the shared buffer arena.  (If your
shared_buffers settings is not somewhere near 100MB, then this theory
breaks down.)

It would be worth using plain old top to watch this process.  We have
enough experience with that to be pretty sure how to interpret its
numbers: "RES minus SHR" is the value to be worried about.

                        regards, tom lane

pgsql-general by date:

Previous
From: Kevin Grittner
Date:
Subject: Re: Understanding Postgres Memory Usage
Next
From: Ahsan Ali
Date:
Subject: LOG: could not fork new process for connection: Cannot allocate memory