Re: Time delayed LR (WAS Re: logical replication restrictions) - Mailing list pgsql-hackers

From Masahiko Sawada
Subject Re: Time delayed LR (WAS Re: logical replication restrictions)
Date
Msg-id CAD21AoDSwQTAMs0J5RW1yRGgrN6vDuqbrBt_4K9U1OzwBVDtGA@mail.gmail.com
Whole thread Raw
In response to RE: Time delayed LR (WAS Re: logical replication restrictions)  ("Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>)
Responses Re: Time delayed LR (WAS Re: logical replication restrictions)  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Wed, Mar 1, 2023 at 6:21 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
>
> Dear Sawada-san,
>
> Thank you for giving your consideration!
>
> > >  We have documented at least one such case
> > > already where during Drop Subscription, if the network is not
> > > reachable then also, a similar problem can happen and users need to be
> > > careful about it [1].
> >
> > Apart from a bad-use case example I mentioned, in general, piling up
> > WAL files due to the replication slot has many bad effects on the
> > system. I'm concerned that the side effect of this feature (at least
> > of the current design) is too huge compared to the benefit, and afraid
> > that users might end up using this feature without understanding the
> > side effect well. It might be okay if we thoroughly document it but
> > I'm not sure.
>
> One approach is that change max_slot_wal_keep_size forcibly when min_send_delay
> is set. But it may lead to disable the slot because WALs needed by the time-delayed
> replication may be also removed. Just the right value cannot be set by us because
> it is quite depends on the min_send_delay and workload.
>
> How about throwing the WARNING when min_send_delay > 0 but
> max_slot_wal_keep_size < 0? Differ from previous, version the subscription
> parameter min_send_delay will be sent to publisher. Therefore, we can compare
> min_send_delay and max_slot_wal_keep_size when publisher receives the parameter.

Since max_slot_wal_keep_size can be changed by reloading the config
file, each walsender warns it also at that time? Not sure it's
helpful. I think it's a legitimate use case to set min_send_delay > 0
and max_slot_wal_keep_size = -1, and users might not even notice the
WARNING message.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: Peter Smith
Date:
Subject: Re: Allow logical replication to copy tables in binary format
Next
From: Tom Lane
Date:
Subject: Re: typedef struct LogicalDecodingContext