Re: Using ProcSignal to get memory context stats from a running backend - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Using ProcSignal to get memory context stats from a running backend
Date
Msg-id CAMsr+YHii-BCC7ddpbb8fpCgzt0wMRt5GYZ0W_kD_Ft8rwWPiQ@mail.gmail.com
Whole thread Raw
In response to Re: Using ProcSignal to get memory context stats from a running backend  (Craig Ringer <craig@2ndquadrant.com>)
Responses Re: Using ProcSignal to get memory context stats from a runningbackend  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 20 December 2017 at 08:46, Craig Ringer <craig@2ndquadrant.com> wrote:
On 20 December 2017 at 02:35, Andres Freund <andres@anarazel.de> wrote:
 
> Yeah.  But please don't mess with MemoryContextStats per se ---
> I dunno about you guys but "call MemoryContextStats(TopMemoryContext)"
> is kinda wired into my gdb reflexes.  I think what'd make sense
> is a new function "MemoryContextStatsTo(context, function_pointer)".
> It's okay to redefine the APIs of the per-context-type functions
> these would call, though, because nobody calls those functions directly.

We already have MemoryContextStatsDetail() - it seems to make sense to
expand that API and leave MemoryContextStats() alone. I don't think
there's a need for a third variant?

Cool, can do.

I'll have to expose a typedef for the printf-wrapper callback in memnodes.h and add it to the stats method, which I thought would be more likely to get complaints than the global hook. I'm actually happier to do it with a passed callback.

Will revise when I get a chance in the next couple of days.

Done.

It uses vfprintf unconditionally, even on Windows where we'd usually use write_stderr, so it doesn't change the current MemoryContextStats behaviour.

2017-12-21 14:39:20.045 AWST [10588] pg_regress/misc_functions LOG:  diagnostic dump requested; memory context info: TopMemoryContext: 67440 total in 5 blocks; 13376 free (6 chunks); 54064 used
          pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 1376 free (0 chunks); 6816 used
          TopTransactionContext: 8192 total in 1 blocks; 7728 free (1 chunks); 464 used
          ...

To verify that stderr output still works properly I ran:


SELECT
  repeat('spamspamspam', 20000000),
  repeat('spamspamspam', 20000000),
  repeat('spamspamspam', 20000000),
  repeat('spamspamspam', 20000000),
  ... lots ...;


and got the expected

+ERROR:  out of memory
+DETAIL:  Failed on request of size 240000004.

and memory context dump to stderr. I didn't add a TAP test for that because I'd rather avoid exercising OOM in tests where we don't know what the host's swap config is like, how its ulimit behaves, whether it has craziness like the OOM killer, etc. But it might make sense to add a TAP test to set a low ulimit and exercise OOM recovery at some point.

I've added a separate vfprintf wrapper for memory context use that doesn't try to use the windows error log like write_stderr(...) does. If we want to change OOM behaviour on Windows it can be done in a followup.
 
--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
Attachment

pgsql-hackers by date:

Previous
From: Rajkumar Raghuwanshi
Date:
Subject: != should give error?
Next
From: Andres Freund
Date:
Subject: Re: Using ProcSignal to get memory context stats from a runningbackend