Re: Skipping logical replication transactions on subscriber side - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Skipping logical replication transactions on subscriber side
Date
Msg-id CAA4eK1+24R3ncmOv67PV7bNV4zjf8VXbe4eh=RBXhXgTd-GcYQ@mail.gmail.com
Whole thread Raw
In response to Re: Skipping logical replication transactions on subscriber side  (Masahiko Sawada <sawada.mshk@gmail.com>)
Responses Re: Skipping logical replication transactions on subscriber side
List pgsql-hackers
On Tue, Oct 19, 2021 at 8:23 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Mon, Oct 18, 2021 at 6:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Mon, Oct 11, 2021 at 1:00 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > On Sun, Oct 10, 2021 at 11:04 PM Peter Eisentraut
> > > <peter.eisentraut@enterprisedb.com> wrote:
> > > >
> > > > On 04.10.21 02:31, Masahiko Sawada wrote:
> > > > > I guess disabling subscriptions on error/conflict and skipping the
> > > > > particular transactions are somewhat different types of functions.
> > > > > Disabling subscriptions on error/conflict seems likes a setting
> > > > > parameter of subscriptions. The users might want to specify this
> > > > > option at creation time. Whereas, skipping the particular transaction
> > > > > is a repair function that the user might want to use on the spot in
> > > > > case of a failure. I’m concerned a bit that combining these functions
> > > > > to one syntax could confuse the users.
> > > >
> > > > Also, would the skip option be dumped and restored using pg_dump?  Maybe
> > > > there is an argument for yes, but if not, then we probably need a
> > > > different path of handling it separate from the more permanent options.
> > >
> > > Good point. I don’t think the skip option should be dumped and
> > > restored using pg_dump since the utilization of transaction ids in
> > > another installation is different.
> > >
> >
> > This is a xid of publisher which subscriber wants to skip. So, even if
> > one restores the subscriber data in a different installation why would
> > it matter till it points to the same publisher?
> >
> > Either way, can't we handle this in pg_dump?
>
> Because of backups (dumps), I think we cannot expect that the user
> restore it somewhere soon. If the dump is restored several months
> later, the publisher could be a different installation (by rebuilding
> from scratch) or XID of the publisher could already be wrapped around.
> It might be useful to dump the skip_xid by pg_dump in some cases, but
> I think it should be optional if we want to do that.
>

Agreed, I think it depends on the use case, so we can keep it
optional, or maybe in the initial version let's not dump it, and only
if we later see the use case then we can add an optional parameter in
pg_dump. Do you think we need any special handling if we decide not to
dump it? I think if we decide to dump it either optionally or
otherwise, then we do need changes in pg_dump.

--
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Data is copied twice when specifying both child and parent table in publication
Next
From: "tanghy.fnst@fujitsu.com"
Date:
Subject: RE: Added schema level support for publication.