Re: Logical replication keepalive flood - Mailing list pgsql-hackers

From Greg Nancarrow
Subject Re: Logical replication keepalive flood
Date
Msg-id CAJcOf-e=5gKc3zQgjw6C8pzu3JB4pENQsAY7QO62rW5x8gpxPQ@mail.gmail.com
Whole thread Raw
In response to Re: Logical replication keepalive flood  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Sun, Sep 19, 2021 at 3:47 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> >
> > Yes, pg_recvlogical seems to be relying on receiving a keepalive for
> > its "--endpos" logic to work (and the 006 test is relying on '' record
> > output from pg_recvlogical in this case).
> > But is it correct to be relying on a keepalive for this?
> >
>
> I don't think this experiment/test indicates that pg_recvlogical's
> "--endpos" relies on keepalive. It would just print the records till
> --endpos and then exit. In the test under discussion, as per
> confirmation by Hou-San, the BEGIN record received has the same LSN as
> --endpos, so it would just output that and exit which is what is
> mentioned in pg_recvlogical docs as well (If there's a record with LSN
> exactly equal to lsn, the record will be output).
>

It seems to be relying on keepalive to get ONE specific record per
--endpos value, because once we apply the
"v1-0002-WIP-skip-the-keepalive-on-FIRST-loop-iteration.patch" patch,
then when pg_recvlogical is invoked for a second time with the same
--endos, it outputs the next record (BEGIN) too. So now for the same
--endpos value, we've had two different records output by
pg_recvlogical.
I have not seen this described in the documentation, so I think it
will need to be updated, should keepalives be reduced as per the
patch. The current documentation seems to be implying that one
particular record will be returned for a given --endpos  (at least,
there is no mention of the possibility of different records being
output for the one --endpos, or that the first_lsn of a transaction is
always equal to end_lsn of the previous transaction).

--endpos=lsn

   In --start mode, automatically stop replication and exit with
normal exit status 0 when receiving reaches the specified LSN. If
specified when not in --start mode, an error is raised.

   If there's a record with LSN exactly equal to lsn, the record will be output.

   The --endpos option is not aware of transaction boundaries and may
truncate output partway through a transaction. Any partially output
transaction will not be consumed and will be replayed again when the
slot is next read from. Individual messages are never truncated.


Regards,
Greg Nancarrow
Fujitsu Australia



pgsql-hackers by date:

Previous
From: A Z
Date:
Subject: Improved PostgreSQL Mathematics Support.
Next
From: "David G. Johnston"
Date:
Subject: Re: Improved PostgreSQL Mathematics Support.