Re: shared-memory based stats collector - v69 - Mailing list pgsql-hackers

From David G. Johnston
Subject Re: shared-memory based stats collector - v69
Date
Msg-id CAKFQuwZhSFdZJ8ZtFwBJ017AYkJBWkoQ6YNUAU3n8_xzLRcLZQ@mail.gmail.com
Whole thread Raw
In response to Re: shared-memory based stats collector - v69  (Andres Freund <andres@anarazel.de>)
Responses Re: shared-memory based stats collector - v69  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Tue, Apr 5, 2022 at 4:16 PM Andres Freund <andres@anarazel.de> wrote:
On 2022-04-05 14:43:49 -0700, David G. Johnston wrote:
> On Tue, Apr 5, 2022 at 2:23 PM Andres Freund <andres@anarazel.de> wrote:

>
> > I guess I should add a paragraph about snapshots / fetch consistency.
> >
>
> I apparently confused/combined the two concepts just now so that would help.

Will add.


Thank you.

On a slightly different track, I took the time to write-up a "Purpose" section for pgstat.c :

It may possibly be duplicating some things written elsewhere as I didn't go looking for similar prior art yet, I just wanted to get thoughts down.  This is the kind of preliminary framing I've been constructing in my own mind as I try to absorb this patch.  I haven't formed an opinion whether the actual user-facing documentation should cover some or all of this instead of the preamble to pgstat.c (which could just point to the docs for prerequisite reading).

David J.

 * Purpose:

 * The PgStat namespace defines an API that facilitates concurrent access
 * to a shared memory region where cumulative statistical data is saved.
 * At shutdown, one of the running system workers will initiate the writing
 * of the data to file. Then, during startup (following a clean shutdown) the
 * Postmaster process will early on ensure that the file is loaded into memory.
 *
 * Each cumulative statistic producing system must construct a PgStat_Kind
 * datum in this file. The details are described elsewhere, but of
 * particular importance is that each kind is classified as having either a
 * fixed number of objects that it tracks, or a variable number.
 *
 * During normal operations, the different consumers of the API will have their
 * accessed managed by the API, the protocol used is determined based upon whether
 * the statistical kind is fixed-numbered or variable-numbered.
 * Readers of variable-numbered statistics will have the option to locally
 * cache the data, while writers may have their updates locally queued
 * and applied in a batch. Thus favoring speed over freshness.
 * The fixed-numbered statistics are faster to process and thus forgo
 * these mechanisms in favor of a light-weight lock.
 *
 * Cumulative in this context means that processes must, for numeric data, send
 * a delta (or change) value via the API which will then be added to the
 * stored value in memory. The system does not track individual changes, only
 * their net effect. Additionally, both due to unclean shutdown or user request,
 * statistics can be reset - meaning that their stored numeric values are returned
 * to zero, and any non-numeric data that may be tracked (say a timestamp) is cleared.


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: shared-memory based stats collector - v70
Next
From: "David G. Johnston"
Date:
Subject: Re: shared-memory based stats collector - v70