Re: Synchronizing slots from primary to standby - Mailing list pgsql-hackers

On Mon, Nov 29, 2021 at 1:10 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Mon, Nov 29, 2021 at 12:19 PM Bharath Rupireddy
> <bharath.rupireddyforpostgres@gmail.com> wrote:
> >
> > On Mon, Nov 29, 2021 at 11:14 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > On Mon, Nov 29, 2021 at 9:40 AM Bharath Rupireddy
> > > <bharath.rupireddyforpostgres@gmail.com> wrote:
> > > >
> > > > On Mon, Nov 29, 2021 at 1:48 AM SATYANARAYANA NARLAPURAM
> > > > <satyanarlapuram@gmail.com> wrote:
> > > > >
> > > > >> 3) Instead of the subscriber pulling the slot info, why can't the
> > > > >> publisher (via the walsender or a new bg worker maybe?) push the
> > > > >> latest slot info? I'm not sure we want to add more functionality to
> > > > >> the walsender, if yes, isn't it going to be much simpler?
> > > > >
> > > > > Standby pulling the information or at least making a first attempt to connect to the  primary is a better
designas primary doesn't need to spend its cycles repeatedly connecting to an unreachable standby. In fact, primary
wouldn'teven need to know the followers, for example followers / log shipping standbys 
> > > >
> > > > My idea was to let the existing walsender from the primary/publisher
> > > > to send the slot info (both logical and physical replication slots) to
> > > > the standby/subscriber, probably by piggybacking the slot info with
> > > > the WAL currently it sends. Having said that, I don't know the
> > > > feasibility of it. Anyways, I'm not in favour of having a new bg
> > > > worker to just ship the slot info. The standby/subscriber, while
> > > > making connection to primary/publisher, can choose to get the
> > > > replication slot info.
> > >
> > > I think it is possible that the standby is restoring the WAL directly
> > > from the archive location and there might not be any wal sender at
> > > time. So I think the idea of standby pulling the WAL looks better to
> > > me.
> >
> > My point was that why can't we let the walreceiver (of course users
> > can configure it on the standby/subscriber) to choose whether or not
> > to receive the replication (both physical and logical) slot info from
> > the primary/publisher and if yes, the walsender(on the
> > primary/publisher) sending it probably as a new WAL record or just
> > piggybacking the replication slot info with any of the existing WAL
> > records.
>
> Okay, I thought your point was that the primary pushing is better over
> standby pulling the slot info, but now it seems that you also agree
> that standby pulling is better right?  Now it appears your point is
> about whether we will use the same connection for pulling the slot
> information which we are using for streaming the data or any other
> connection?  I mean in this patch also we are creating a replication
> connection and pulling the slot information over there, just point is
> we are starting a separate worker for pulling the slot information,
> and I think that approach is better as this will not impact the
> performance of the other replication connection which we are using for
> communicating the data.

The easiest way to implement this feature so far, is to use a common
bg worker (as opposed to the bg worker proposed originally in this
thread which, IIUC, works for logical replication) running on standby
(in case of streaming replication with physical replication slots) or
subscriber (in case of logical replication with logical replication
slots) for getting both the physical and logical replication slots
info from the primary or publisher. This bg worker requires at least
two GUCs, 1) to enable/disable the worker 2) to define the slot sync
interval (the bg worker gets the slots info after every sync interval
of time).

Thoughts?

Regards,
Bharath Rupireddy.