Thread: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

On Wed, Apr 20, 2022 at 04:40:44PM -0700, David G. Johnston wrote:
> Hackers,
> 
> The new cumulative stats subsystem no longer has a "lost under heavy load"
> problem so that parenthetical should go (or at least be modified).
> 
> These stats can be reset so some discussion about how the system uses them
> given that possibility seems like it would be good to add here.  I'm not sure
> what that should look like though.
> 
> diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> index 04a04e0e5f..360807c8f9 100644
> --- a/doc/src/sgml/maintenance.sgml
> +++ b/doc/src/sgml/maintenance.sgml
> @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> vacuum insert scale fac
>      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
>      the number of inserted tuples are obtained from the cumulative statistics
> system;
>      it is a semi-accurate count updated by each <command>UPDATE</command>,
> -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> -    only semi-accurate because some information might be lost under heavy
> -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> +    <command>DELETE</command> and <command>INSERT</command> operation.
> +    If the <structfield>relfrozenxid</structfield> value of the table
>      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
>      an aggressive vacuum is performed to freeze old tuples and advance
>      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> been modified

Yes, I agree and plan to apply this patch soon.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson

Hi,

I had missed David's original email on this topic...

On 2022-07-14 18:58:09 -0400, Bruce Momjian wrote:
> On Wed, Apr 20, 2022 at 04:40:44PM -0700, David G. Johnston wrote:
> > The new cumulative stats subsystem no longer has a "lost under heavy load"
> > problem so that parenthetical should go (or at least be modified).
> > 
> > These stats can be reset so some discussion about how the system uses them
> > given that possibility seems like it would be good to add here.  I'm not sure
> > what that should look like though.
> > 
> > diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> > index 04a04e0e5f..360807c8f9 100644
> > --- a/doc/src/sgml/maintenance.sgml
> > +++ b/doc/src/sgml/maintenance.sgml
> > @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> > vacuum insert scale fac
> >      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
> >      the number of inserted tuples are obtained from the cumulative statistics
> > system;
> >      it is a semi-accurate count updated by each <command>UPDATE</command>,
> > -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> > -    only semi-accurate because some information might be lost under heavy
> > -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> > +    <command>DELETE</command> and <command>INSERT</command> operation.
> > +    If the <structfield>relfrozenxid</structfield> value of the table
> >      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
> >      an aggressive vacuum is performed to freeze old tuples and advance
> >      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> > been modified
> 
> Yes, I agree and plan to apply this patch soon.

It might make sense to still say semi-accurate, but adjust the explanation to
say that stats reporting is not instantaneous?

Greetings,

Andres Freund

Hi,

On 2022-07-18 19:47:39 -0700, David G. Johnston wrote:
> On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
> > It might make sense to still say semi-accurate, but adjust the explanation
> > to
> > say that stats reporting is not instantaneous?
> >
> >
> Unless that delay manifests in executing an UPDATE in a session then
> looking at these views in the same session and not seeing that update
> reflected I wouldn't mention it.

Depending on which stats you're looking at, yes, that could totally happen. I
don't think the issue is not seeing changes from the current transaction
though - it's that *after* commit you might not see them for a while (the're
transmitted not more than once a second, and can be delayed up to 60s if
there's contention).

Greetings,

Andres Freund

On Mon, Jul 18, 2022 at 08:04:12PM -0700, Andres Freund wrote:
> Hi,
> 
> On 2022-07-18 19:47:39 -0700, David G. Johnston wrote:
> > On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
> > > It might make sense to still say semi-accurate, but adjust the explanation
> > > to
> > > say that stats reporting is not instantaneous?
> > >
> > >
> > Unless that delay manifests in executing an UPDATE in a session then
> > looking at these views in the same session and not seeing that update
> > reflected I wouldn't mention it.
> 
> Depending on which stats you're looking at, yes, that could totally happen. I
> don't think the issue is not seeing changes from the current transaction
> though - it's that *after* commit you might not see them for a while (the're
> transmitted not more than once a second, and can be delayed up to 60s if
> there's contention).

So the docs don't need any changes, I assume.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson

On Fri, Aug 12, 2022 at 12:58:28PM -0700, David G. Johnston wrote:
> I dislike using the word accurate here now, it will be accurate, but we don't
> promise perfect timeliness.  So it needs to change:
> 
>  diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> index 04a04e0e5f..360807c8f9 100644
> --- a/doc/src/sgml/maintenance.sgml
> +++ b/doc/src/sgml/maintenance.sgml
> @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> vacuum insert scale fac
>      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
>      the number of inserted tuples are obtained from the cumulative statistics
> system;
>      it is a semi-accurate count updated by each <command>UPDATE</command>,
> -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> -    only semi-accurate because some information might be lost under heavy
> -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> +    <command>DELETE</command> and <command>INSERT</command> operation.
> +    If the <structfield>relfrozenxid</structfield> value of the table
>      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
>      an aggressive vacuum is performed to freeze old tuples and advance
>      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> been modified

Done in master.

> However, also replace the remaining instance of "a semi-accurate count" with
> "an eventually-consistent count".

> ...it is an eventually-consistent count updated by each UPDATE, DELETE, and
> INSERT operation.

Also done.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.