Re: Logical Replication of sequences - Mailing list pgsql-hackers
| From | Ashutosh Bapat |
|---|---|
| Subject | Re: Logical Replication of sequences |
| Date | |
| Msg-id | CAExHW5vus=YPKe8fjLdQNqOzTKjShYiOr65UK1C=PhasEisNRg@mail.gmail.com Whole thread Raw |
| In response to | Re: Logical Replication of sequences (Amit Kapila <amit.kapila16@gmail.com>) |
| Responses |
Re: Logical Replication of sequences
|
| List | pgsql-hackers |
On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Tue, Jun 4, 2024 at 7:40 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> On Tue, Jun 4, 2024 at 4:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>>
>>
>> 3. Replicate published sequences via walsender at the time of shutdown
>> or incrementally while decoding checkpoint record. The two ways to
>> achieve this are: (a) WAL log a special NOOP record just before
>> shutting down checkpointer. Then allow the WALsender to read the
>> sequence data and send it to the subscriber while decoding the new
>> NOOP record. (b) Similar to the previous idea but instead of WAL
>> logging a new record directly invokes a decoding callback after
>> walsender receives a request to shutdown which will allow pgoutput to
>> read and send required sequences. This approach has a drawback that we
>> are adding more work at the time of shutdown but note that we already
>> waits for all the WAL records to be decoded and sent before shutting
>> down the walsender during shutdown of the node.
>>
>> Any other ideas?
>>
>
> In case of primary crash the sequence won't get replicated. That is true even with the previous approach in case walsender is shut down because of a crash, but it is more serious with this approach.
>
Right, but if we just want to support a major version upgrade scenario
then this should be fine because upgrades require a clean shutdown.
>
How about periodically sending this information?
>
Now, if we want to support some sort of failover then probably this
will help. Do you have that use case in mind?
Regular failover was a goal for supporting logical replication of sequences. That might be more common than major upgrade scenario.
If we want to send
periodically then we can do it when decoding checkpoint
(XLOG_CHECKPOINT_ONLINE) or some other periodic WAL record like
running_xacts (XLOG_RUNNING_XACTS).
Yeah. I am thinking along those lines.
It must be noted, however, that none of those optional make sure that the replicated sequence's states are consistent with the replicated object state which use those sequences. E.g. table t1 uses sequence s1. By last sequence replication, as of time T1, let's say t1 had consumed values upto vl1 from s1. But later, by time T2, it consumed values upto vl2 which were not replicated but the changes to t1 by T2 were replicated. If failover happens at that point, INSERTs on t1 would fail because of duplicate keys (values between vl1 and vl2). Previous attempt to support logical sequence replication solved this problem by replicating a future state of sequences (current value +/- log count). Similarly, if the sequence was ALTERed between T1 and T2, the state of sequence on replica would be inconsistent with the state of t1. Failing over at this stage, might end t1 in an inconsistent state.
Best Wishes,
Ashutosh Bapat
pgsql-hackers by date: