Re: Report bytes and transactions actually sent downtream - Mailing list pgsql-hackers

On Fri, Sep 26, 2025 at 4:43 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
>
>
> === 2
>
> Playing with "has stats" a bit.
>
> -- Issue 1:
>

Thanks for experiments! Thanks for bringing it up.

> Say, plugin has stats enabled and I get:
>
> postgres=# select plugin,plugin_sent_txns from pg_stat_replication_slots ;
>      plugin     | plugin_sent_txns
> ----------------+------------------
>  pg_commit_info |                9
> (1 row)
>
> If the engine is shutdown and the plugin is now replaced by a version that
> does not provide stats, then, right after startup, I still get:
>
> postgres=# select plugin,plugin_sent_txns from pg_stat_replication_slots ;
>      plugin     | plugin_sent_txns
> ----------------+------------------
>  pg_commit_info |                9
> (1 row)
>
> And that will be the case until the plugin decodes something (so that
> statent->plugin_has_stats gets replaced in pgstat_report_replslot()).
>
> That's because plugin_has_stats is stored in PgStat_StatReplSlotEntry
> and so it's restored from the stat file when the engine starts.
>
> Now, let's do some inserts and decode:
>
> postgres=# insert into t1 values ('a');
> INSERT 0 1
> postgres=# insert into t1 values ('a');
> INSERT 0 1
> postgres=# select * from pg_logical_slot_get_changes('logical_slot',NULL,NULL);
>     lsn     | xid |                                          data
> ------------+-----+-----------------------------------------------------------------------------------------
>  0/407121C0 | 766 | xid 766: lsn:0/40712190 inserts:1 deletes:0 updates:0 truncates:0 relations truncated:0
>  0/40712268 | 767 | xid 767: lsn:0/40712238 inserts:1 deletes:0 updates:0 truncates:0 relations truncated:0
> (2 rows)
>
> postgres=# select plugin,plugin_sent_txns from pg_stat_replication_slots ;
>      plugin     | plugin_sent_txns
> ----------------+------------------
>  pg_commit_info |
> (1 row)
>
> All good.
>
> Issue 1 is that before any decoding happens, pg_stat_replication_slots is still
> showing stale plugin statistics from a plugin that may no longer support stats.
>
> I'm not sure how we could easily fix this issue, as we don't know the plugin's
> stats capability until we actually use it.
>

I don't think this is an issue. There is no way for the core to tell
whether the plugin will provide stats or not, unless it sets that
ctx->stats which happens in the startup callback. Till then it is
rightly providing the values accumulated so far. Once the decoding
starts, we know that the plugin is not providing any stats and we
don't display anything.

> -- Issue 2:
>
> Let's shutdown, replace the plugin with a version that has stats enabled and
> restart.
>
> Same behavior as before:
>
> postgres=# select plugin,plugin_sent_txns from pg_stat_replication_slots ;
>      plugin     | plugin_sent_txns
> ----------------+------------------
>  pg_commit_info |
> (1 row)
>
> Until pgstat_report_replslot() is not called, the statent->plugin_has_stats is
> not updated. So it displays the stats as they were before the shutdown. But that's
> not an issue in this case (when switching from non stats to stats).
>
> Now, let's do some inserts and decode:
>
> postgres=# insert into t1 values ('a');
> INSERT 0 1
> postgres=# select * from pg_logical_slot_get_changes('logical_slot',NULL,NULL);
>     lsn     | xid |                                          data
> ------------+-----+-----------------------------------------------------------------------------------------
>  0/407125B0 | 768 | xid 768: lsn:0/40712580 inserts:1 deletes:0 updates:0 truncates:0 relations truncated:0
> (1 row)
>
> and check the stats:
>
> postgres=# select plugin,plugin_sent_txns from pg_stat_replication_slots ;
>      plugin     | plugin_sent_txns
> ----------------+------------------
>  pg_commit_info |               10
> (1 row)
>
> Now it reports 10, that's the 9 before we changed the plugin to not have stats
> enabled plus this new one.
>
> Issue 2: when switching from a non-stats plugin back to a stats-capable plugin, it
> shows accumulated values from before the non-stats switch.

This too seems to be a non-issue to me. The stats in the view get
reset only when a user resets them. So we shouldn't wipe out the
already accumulated values just because the plugin stopped providing
it. If the plugin keeps flip-flopping and only partial statistics
provided by the plugin will be accumulated. That's the plugin's
responsibility. Realistically a plugin will either decide to provide
statistics in some version and then continue forever OR it will decide
against it. Flip-flopping won't happen in practice.

If at all we decide to reset the stats when the plugin does not
provide them, I think a better fix is to set them to 0 in
pgstat_report_replslot() independent of previous state of has_stats.
It will be more or less same CPU instructions. like below
if (repSlotStat->plugin_has_stats)
{
REPLSLOT_ACC(plugin_sent_txns);
REPLSLOT_ACC(plugin_sent_bytes);
REPLSLOT_ACC(plugin_filtered_bytes);
}
else
{
statent->plugin_sent_txns = 0;
statent->plugin_sent_bytes = 0;
statent->plugin_filtered_bytes = 0
}

--
Best Wishes,
Ashutosh Bapat