Re: Time delayed LR (WAS Re: logical replication restrictions) - Mailing list pgsql-hackers

From Önder Kalacı
Subject Re: Time delayed LR (WAS Re: logical replication restrictions)
Date
Msg-id CACawEhVhQPh3+gdYiO0k1JHBY71Si8dd0WXWsYz8WLiybRs3tw@mail.gmail.com
Whole thread Raw
In response to Re: Time delayed LR (WAS Re: logical replication restrictions)  (Masahiko Sawada <sawada.mshk@gmail.com>)
Responses Re: Time delayed LR (WAS Re: logical replication restrictions)  (Nathan Bossart <nathandbossart@gmail.com>)
List pgsql-hackers
Hi all,

Thanks for working on this.


I imagine that a typical use case would be to set min_send_delay to
several hours to days. I'm concerned that it could not be an
acceptable trade-off for many users that the system cannot collect any
garbage during that.

I'm not too worried about the WAL recycling, that mostly looks like
a documentation issue to me. It is not a problem that many PG users
are unfamiliar. Also, even though one day creating - altering subscription
is relaxed to be done by a regular user, one option could be to require
this setting to be changed by a superuser? That would alleviate my concern
regarding WAL recycling. A superuser should be able to monitor the system
and adjust the settings/hardware accordingly.

However, VACUUM being blocked by replication with a configuration
change on the subscription sounds more concerning to me. Blocking
VACUUM for hours could quickly escalate to performance problems.

On the other hand, we already have a similar problem with 
recovery_min_apply_delay combined with hot_standby_feedback [1].
So, that probably is an acceptable trade-off for the pgsql-hackers.
If you use this feature, you should be even more careful.


I think we can have the apply process write the decoded changes
somewhere on the disk (as not temporary files) and return the flush
LSN so that the apply worker can apply them later and the publisher
can advance slot's LSN. The feature would be more complex but from the
user perspective it would be better.

Yes, this might probably be one of the ideal solutions to the problem at hand. But, 
my current guess is that it'd be a non-trivial change with different concurrency/failure
scenarios. So, I'm not sure if that is going to be a realistic patch to pursue.


Thanks,
Onder KALACI 

 

 

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: a very minor bug and a couple of comment changes for basebackup.c
Next
From: Melanie Plageman
Date:
Subject: Re: Track IO times in pg_stat_io