Re: Get memory contexts of an arbitrary backend process - Mailing list pgsql-hackers

On 2021-03-18 15:09, Fujii Masao wrote:

Thanks for your comments!

> On 2021/03/17 22:24, torikoshia wrote:
>> I remade the patch and introduced a function
>> pg_print_backend_memory_contexts(PID) which prints the memory contexts 
>> of
>> the specified PID to elog.
> 
> Thanks for the patch!
> 
> 
>>    =# SELECT pg_print_backend_memory_contexts(450855);
>> 
>>    ** log output **
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  Printing memory contexts 
>> of PID 450855
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 0 
>> TopMemoryContext: 68720 total in 5 blocks; 16312 free (15 chunks); 
>> 52408 used
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 Prepared 
>> Queries: 65536 total in 4 blocks; 35088 free (14 chunks); 30448 used
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 pgstat 
>> TabStatusArray lookup hash table: 8192 total in 1 blocks; 1408 free (0 
>> chunks); 6784 used
>>    ..(snip)..
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 2 
>> CachedPlanSource: 4096 total in 3 blocks; 680 free (0 chunks); 3416 
>> used: PREPARE hoge_200 AS SELECT * FROM pgbench_accounts WHERE aid = 
>> 1111111111111111111111111111111111111...
>>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 3 
>> CachedPlanQuery: 4096 total in 3 blocks; 464 free (0 chunks); 3632 
>> used
>>    ..(snip)..
>>    2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 Timezones: 
>> 104128 total in 2 blocks; 2584 free (0 chunks); 101544 used
>>    2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 ErrorContext: 
>> 8192 total in 1 blocks; 7928 free (5 chunks); 264 used
>>    2021-03-17 15:21:01.945 JST [450855] LOG:  Grand total: 2802080 
>> bytes in 1399 blocks; 480568 free (178 chunks); 2321512 used
>> 
>> 
>> As above, the output is almost the same as MemoryContextStatsPrint()
>> except for the way of expression of the level.
>> MemoryContextStatsPrint() uses indents, but
>> pg_print_backend_memory_contexts() writes it as "level: %d".
> 
> This format looks better to me.
> 
> 
>> Since there was discussion about enlarging StringInfo may cause
>> errors on OOM[1], this patch calls elog for each context.
>> 
>> As with MemoryContextStatsPrint(), each context shows 100
>> children at most.
>> I once thought it should be configurable, but something like
>> pg_print_backend_memory_contexts(PID, num_children) needs to send
>> the 'num_children' from requestor to dumper and it seems to require
>> another infrastructure.
>> Creating a new GUC for this seems overkill.
>> If MemoryContextStatsPrint(), i.e. showing 100 children at most is
>> enough, this hard limit may be acceptable.
> 
> Can't this number be passed via shared memory?

The attached patch uses static shared memory to pass the number.

As documented, the current implementation allows that when multiple
pg_print_backend_memory_contexts() called in succession or
simultaneously, max_children can be the one of another
pg_print_backend_memory_contexts().

I had tried to avoid this by adding some state information and using
before_shmem_exit() in case of process termination for cleaning up the
state information as in the patch I presented earlier, but since kill()
returns success before the dumper called signal handler, it seemed
there were times when we couldn't clean up the state.

Since this happens only when multiple pg_print_backend_memory_contexts()
are called and their specified number of children are different, and the
effect is just the not intended number of children to print, it might be
acceptable.

Or it might be better to wait for some seconds if num_chilren on shared
memory is not the initialized value(meaning some other process is
requesting to print memory contexts).

>> Only superusers can call pg_print_backend_memory_contexts().
> 
> +    /* Only allow superusers to signal superuser-owned backends. */
> +    if (superuser_arg(proc->roleId) && !superuser())
> 
> The patch seems to allow even non-superuser to request to print the 
> memory
> contexts if the target backend is owned by non-superuser. Is this 
> intentional?
> I think that only superuser should be allowed to execute
> pg_print_backend_memory_contexts() whoever owns the target backend.
> Because that function can cause lots of log messages.

Thanks, it's not intentional, modified it.

>> I'm going to add documentation and regression tests.

Added them.

Regards,