Re: Enhancing Memory Context Statistics Reporting - Mailing list pgsql-hackers

From Rahila Syed
Subject Re: Enhancing Memory Context Statistics Reporting
Date
Msg-id CAH2L28uayhv+AxgPLThexJ21NA8j7XFiYqu6rgsZSSNosvPjvg@mail.gmail.com
Whole thread Raw
In response to Re: Enhancing Memory Context Statistics Reporting  (Tomas Vondra <tomas@vondra.me>)
List pgsql-hackers

I think something is not quite right, because if I try running a simple
pgbench script that does pg_get_process_memory_contexts() on PIDs of
random postgres process (just like in the past), I immediately get this:

Thank you for testing. This issue occurs when a process that previously attached
to a DSA area for publishing its own context statistics tries to attach to it again while
querying statistics from another backend. Previously, I was not detaching at the end
of publishing the statistics. I have now changed it to detach from the area after the
statistics are published. The fix is included in the updated patch.
 
Perhaps the backends need to synchronize creation of the DSA?

This has been implemented in the patch.
 
Sounds good. Do you have any measurements how much this reduced the size
of the entries written to the DSA? How many entries will fit into 1MB of
shared memory?

The size of the entries has approximately halved after dynamically allocating the 
strings and a datum array.
Also, previously, I was allocating the entire memory for all contexts in one large chunk
from DSA. I have now separated them into smaller allocations 
per context. The integer counters are still allocated at once for all contexts, but  
the size of an allocated chunk will not exceed approximately 128 bytes * total_num_of_contexts.
Average total number of contexts is in the hundreds.

PFA the updated and rebased patches.

Thank you,
Rahila Syed

Attachment

pgsql-hackers by date:

Previous
From: Ilia Evdokimov
Date:
Subject: Re: a very significant fraction of the buildfarm is now pink
Next
From: Bertrand Drouvot
Date:
Subject: Re: Remove wal_[sync|write][_time] from pg_stat_wal and track_wal_io_timing