Re: Time delayed LR (WAS Re: logical replication restrictions) - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Time delayed LR (WAS Re: logical replication restrictions)
Date
Msg-id CAA4eK1+aKKfxs25W_T=0=T2iBMxVfZ2+CjdHDrVsUh6LDn37qw@mail.gmail.com
Whole thread Raw
In response to Re: Time delayed LR (WAS Re: logical replication restrictions)  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Fri, May 12, 2023 at 10:33 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, May 12, 2023 at 7:38 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Thu, May 11, 2023 at 2:04 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > On Fri, Apr 28, 2023 at 2:35 PM Hayato Kuroda (Fujitsu)
> > > <kuroda.hayato@fujitsu.com> wrote:
> > > >
> > > > Dear hackers,
> > > >
> > > > I rebased and refined my PoC. Followings are the changes:
> > > >
> > >
> > > 1. Is my understanding correct that this patch creates the delay files
> > > for each transaction? If so, did you consider other approaches such as
> > > using one file to avoid creating many files?
> > > 2. For streaming transactions, first the changes are written in the
> > > temp file and then moved to the delay file. It seems like there is a
> > > double work. Is it possible to unify it such that when min_apply_delay
> > > is specified, we just use the delay file without sacrificing the
> > > advantages like stream sub-abort can truncate the changes?
> > > 3. Ideally, there shouldn't be a performance impact of this feature on
> > > regular transactions because the delay file is created only when
> > > min_apply_delay is active but better to do some testing of the same.
> > >
> >
> > In addition to the points Amit raised, if the 'required_schema' option
> > is specified in START_REPLICATION, the publisher sends schema
> > information for every change. I think it leads to significant
> > overhead. Did you consider alternative approaches such as sending the
> > schema information for every transaction or the subscriber requests
> > the publisher to send it?
> >
>
> Why do we need this new flag? I can't see any comments in the related
> code which explain its need.
>

So as per the email [1], this would be required after the subscriber
restart. I guess we ideally need it once per delay file (considering
that we have one file for all delayed xacts). In the worst case, we
can have it per transaction as suggested by Sawada-San.

[1] -
https://www.postgresql.org/message-id/TYAPR01MB5866568A5C1E71338328B20CF5629%40TYAPR01MB5866.jpnprd01.prod.outlook.com

--
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Time delayed LR (WAS Re: logical replication restrictions)
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: pg_stat_io for the startup process