Re: Replication slot stats misgivings - Mailing list pgsql-hackers

On Mon, Apr 12, 2021 at 9:36 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Apr 12, 2021 at 5:29 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Mon, Apr 12, 2021 at 8:08 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > On Mon, Apr 12, 2021 at 4:34 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > >
> > > > On Mon, Apr 12, 2021 at 6:19 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > >
> > > > > On Mon, Apr 12, 2021 at 10:27 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > > > >
> > > > > > On Sat, Apr 10, 2021 at 9:53 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > > It seems Vignesh has changed patches based on the latest set of
> > > > > > > comments so you might want to rebase.
> > > > > >
> > > > > > I've merged my patch into the v6 patch set Vignesh submitted.
> > > > > >
> > > > > > I've attached the updated version of the patches. I didn't change
> > > > > > anything in the patch that changes char[NAMEDATALEN] to NameData (0001
> > > > > > patch) and patches that add tests.
> > > > > >
> > > > >
> > > > > I think we can push 0001. What do you think?
> > > >
> > > > +1
> > > >
> > > > >
> > > > > > In 0003 patch I reordered the
> > > > > > output parameters of pg_stat_replication_slots; showing total number
> > > > > > of transactions and total bytes followed by statistics for spilled and
> > > > > > streamed transactions seems appropriate to me.
> > > > > >
> > > > >
> > > > > I am not sure about this because I think we might want to add some
> > > > > info of stream/spill bytes in total_bytes description (something like
> > > > > stream/spill bytes are not in addition to total_bytes).
> >
> > BTW doesn't it confuse users that stream/spill bytes are not in
> > addition to total_bytes? User will need to do "total_bytes +
> > spill/stream_bytes" to know the actual total amount of data sent to
> > the decoding output plugin, is that right?
> >
>
> No, total_bytes includes the spill/stream bytes. So, the user doesn't
> need to do any calculation to compute totel_bytes sent to output
> plugin.

The following test for the latest v8 patch seems to show different.
total_bytes is 1808 whereas spill_bytes is 13200000. Am I missing
something?

postgres(1:85969)=# select pg_create_logical_replication_slot('s',
'test_decoding');
 pg_create_logical_replication_slot
------------------------------------
 (s,0/1884468)
(1 row)

postgres(1:85969)=# create table a (i int);
CREATE TABLE
postgres(1:85969)=# insert into a select generate_series(1, 100000);
INSERT 0 100000
postgres(1:85969)=# set logical_decoding_work_mem to 64;
SET
postgres(1:85969)=# select * from pg_stat_replication_slots ;
 slot_name | total_txns | total_bytes | spill_txns | spill_count |
spill_bytes | stream_txns | stream_count | stream_bytes | stats_reset

-----------+------------+-------------+------------+-------------+-------------+-------------+--------------+--------------+-------------
 s         |          0 |           0 |          0 |           0 |
      0 |           0 |            0 |            0 |
(1 row)

postgres(1:85969)=# select count(*) from
pg_logical_slot_peek_changes('s', NULL, NULL);
 count
--------
 100004
(1 row)

postgres(1:85969)=# select * from pg_stat_replication_slots ;
 slot_name | total_txns | total_bytes | spill_txns | spill_count |
spill_bytes | stream_txns | stream_count | stream_bytes | stats_reset

-----------+------------+-------------+------------+-------------+-------------+-------------+--------------+--------------+-------------
 s         |          2 |        1808 |          1 |         202 |
13200000 |           0 |            0 |            0 |
(1 row)

Regards,

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/