Re: Skip unregistered custom kinds on stats load - Mailing list pgsql-hackers

From Sami Imseih
Subject Re: Skip unregistered custom kinds on stats load
Date
Msg-id CAA5RZ0tGsNSHzY9wX=0zz5Ej_DCCSjUBXpNQskT8eXThhOSETQ@mail.gmail.com
Whole thread Raw
In response to Re: Skip unregistered custom kinds on stats load  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Skip unregistered custom kinds on stats load
List pgsql-hackers
> On Tue, Oct 21, 2025 at 01:26:30PM -0500, Sami Imseih wrote:
> > Hmm, since the pgstat.stat file is managed by core and not by the
> > extension, I think this should be handled semi-automatically by
> > pgstats. Even with the checks you mention above, validating that we
> > are indeed loading the same kind will never be totally foolproof. I
> > think we can validate kinds better by adding some MAGIC NUMBER that is
> > set by the extension during registration and tracked in pgstats.stat.
> > If the kind ID
> > and MAGIC NUMBER (for example, FORMAT ID) match, then we have more of
> > a guarantee that this is the same extension. Also, this is probably a
> > good idea to support extension upgrades.
>
> Probably.  Still, it does not feel like something we need to push
> strongly in a given direction, because we may pay in flexibility
> compared to what's on HEAD.
>
> With a single kind ID forcing how large a fixed-sized entry or one
> variable-sized entry should be based on what kind of data is loaded,
> versioning something based on a single kind is impossible (or the same
> size is used, leading to different contents translated).  Of course,
> one could use multiple kind IDs if they want to enforce different
> stats versions in a single major version of the backend.  To me, this
> property comes down that the version of the pgstats file cannot change
> in a single major version, implying that the same rule is also pushed
> to the custom kinds: if you begin to use a kind ID in a given major
> version, do not change the stats format, limitting new changes to a
> new major version.  Or an upgrade format path to use is something an
> extension needs to handle on its own.  I'd bet that stats upgrade
> requirements for the same major version are not going to be that
> common, the future may tell a different story depending on the use
> cases that pop.

OK, fair enough. I don't have enough compelling use-cases to push
back harder on this. Initially, my focus was on the fact that unlike
built-in stats, custom stats may not have a registered kind and
disposing of all the stats in that case did not seem like the appropriate
behavior, or least too strict of a behavior.

But also, stats themselves are for the most part are consumed
in a way that can tolerate stats being reset, because most of the
time a user is taking deltas of stats between snapshots.

The autovacuum case is special, and the work that Bertrand is
doing will be beneficial to ensure that a/v has the correct stats
after crash recovery.

--

Sami



pgsql-hackers by date:

Previous
From: Sami Imseih
Date:
Subject: [Proposal] Adding callback support for custom statistics kinds
Next
From: Alexander Lakhin
Date:
Subject: Re: Instability of phycodorus in pg_upgrade tests with JIT