Thread: pgsql: Add function to get memory context stats for processes
Add function to get memory context stats for processes This adds a function for retrieving memory context statistics and information from backends as well as auxiliary processes. The intended usecase is cluster debugging when under memory pressure or unanticipated memory usage characteristics. When calling the function it sends a signal to the specified process to submit statistics regarding its memory contexts into dynamic shared memory. Each memory context is returned in detail, followed by a cumulative total in case the number of contexts exceed the max allocated amount of shared memory. Each process is limited to use at most 1Mb memory for this. A summary can also be explicitly requested by the user, this will return the TopMemoryContext and a cumulative total of all lower contexts. In order to not block on busy processes the caller specifies the number of seconds during which to retry before timing out. In the case where no statistics are published within the set timeout, the last known statistics are returned, or NULL if no previously published statistics exist. This allows dash- board type queries to continually publish even if the target process is temporarily congested. Context records contain a timestamp to indicate when they were submitted. Author: Rahila Syed <rahilasyed90@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/042a66291b04f473cbc72f95f07438abd75ae3a9 Modified Files -------------- doc/src/sgml/func.sgml | 172 +++++++ src/backend/catalog/system_views.sql | 5 + src/backend/postmaster/autovacuum.c | 4 + src/backend/postmaster/checkpointer.c | 4 + src/backend/postmaster/interrupt.c | 4 + src/backend/postmaster/pgarch.c | 4 + src/backend/postmaster/startup.c | 4 + src/backend/postmaster/walsummarizer.c | 4 + src/backend/storage/ipc/ipci.c | 3 + src/backend/storage/ipc/procsignal.c | 3 + src/backend/storage/lmgr/lwlock.c | 2 + src/backend/storage/lmgr/proc.c | 1 + src/backend/tcop/postgres.c | 3 + src/backend/utils/activity/wait_event_names.txt | 1 + src/backend/utils/adt/mcxtfuncs.c | 426 ++++++++++++++-- src/backend/utils/init/globals.c | 1 + src/backend/utils/init/postinit.c | 7 + src/backend/utils/mmgr/mcxt.c | 645 +++++++++++++++++++++++- src/include/catalog/pg_proc.dat | 10 + src/include/miscadmin.h | 1 + src/include/storage/lwlock.h | 2 + src/include/storage/procsignal.h | 1 + src/include/utils/memutils.h | 82 +++ src/test/regress/expected/sysviews.out | 19 + src/test/regress/sql/sysviews.sql | 18 + src/tools/pgindent/typedefs.list | 4 + 26 files changed, 1385 insertions(+), 45 deletions(-)
On Tue, 2025-04-08 at 09:10 +0000, Daniel Gustafsson wrote: > Add function to get memory context stats for processes > > This adds a function for retrieving memory context statistics > and information from backends as well as auxiliary processes. > The intended usecase is cluster debugging when under memory > pressure or unanticipated memory usage characteristics. > > Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com > > Details > ------- > https://git.postgresql.org/pg/commitdiff/042a66291b04f473cbc72f95f07438abd75ae3a9 > > [from the patch:] > diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h > index 016dfd9b3f6..cfe14631445 100644 > --- a/src/include/storage/procsignal.h > +++ b/src/include/storage/procsignal.h > [...] > +extern dsa_area *area; This commit causes problems for PostGIS, because the name "area" collides with a PostGIS object: postgis_legacy.c:58:28: error: ‘area’ redeclared as different kind of symbol 58 | POSTGIS_DEPRECATE("3.0.0", area) | ^~~~ postgis_legacy.c:40:15: note: in definition of macro ‘POSTGIS_DEPRECATE’ 40 | Datum funcname(PG_FUNCTION_ARGS); \ | ^~~~~~~~ In file included from ../libpgcommon/lwgeom_pg.h:24, from postgis_legacy.c:37: /home/laurenz/pg/include/postgresql/server/utils/memutils.h:403:18: note: previous declaration of ‘area’ with type ‘dsa_area*’ 403 | extern dsa_area *area; | ^~~~ Now one can take the position that PostGIS as dependent library hs to adapt, but I think "area" is too generic a name. Could you envision renaming the global variable to something like "shm_area"? Attached is a patch for this change. I am not wedded to the name at all, it was just the first thing that popped into my head. Yours, Laurenz Albe
Attachment
Hi, > Now one can take the position that PostGIS as dependent library hs to > adapt, but I think "area" is too generic a name. Could you envision > renaming the global variable to something like "shm_area"? > > Attached is a patch for this change. > I am not wedded to the name at all, it was just the first thing that > popped into my head. I agree that the name is too generic for an exported symbol. -- Best regards, Aleksander Alekseev
> On 10 Apr 2025, at 13:42, Laurenz Albe <laurenz.albe@cybertec.at> wrote: > On Tue, 2025-04-08 at 09:10 +0000, Daniel Gustafsson wrote: >> +extern dsa_area *area; > > This commit causes problems for PostGIS, because the name "area" collides > with a PostGIS object: Thanks for the report, I've already posted a patch [0] and will push that shortly. -- Daniel Gustafsson [0] https://postgr.es/m/68638ACC-3556-429E-93A0-189F73D0E274@yesql.se
On Thu, 2025-04-10 at 14:46 +0200, Daniel Gustafsson wrote: > > On 10 Apr 2025, at 13:42, Laurenz Albe <laurenz.albe@cybertec.at> wrote: > > > +extern dsa_area *area; > > > > This commit causes problems for PostGIS, because the name "area" collides > > with a PostGIS object: > > Thanks for the report, I've already posted a patch [0] and will push that > shortly. Great, thanks. Yours, Laurenz Albe