Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs - Mailing list pgsql-hackers
| From | Chris Travers |
|---|---|
| Subject | Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs |
| Date | |
| Msg-id | CAN-RpxDmJ_agTqWNnfZ5125iXrtHuy-t+kQapu7JTvVQ0Bn3pA@mail.gmail.com Whole thread Raw |
| In response to | Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs (Michael Paquier <michael.paquier@gmail.com>) |
| Responses |
Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs
|
| List | pgsql-hackers |
On Mon, Oct 30, 2017 at 10:57 AM, Michael Paquier <michael.paquier@gmail.com> wrote:
On Mon, Oct 30, 2017 at 9:43 AM, Chris Travers <chris.travers@adjust.com> wrote:
> Are there any cases right now where you have features added by extensions that write to directories which are required for a rewind?
In some of the stuff I maintain, I actually have one case now of a
configuration file included with include_if_exists by postgresql.conf
and is expected to be generated by a component that my code doing the
rewind has no direct access on... I can control how pg_rewind kicks
in, but I think that you would actually break silently the current
code, which is scary especially if I don't control the code where
pg_rewind is called when Postgres 11 gets integrated into the product
I am thinking about, and even more scary if there is no way to include
something.
Ok, so there is an argument that there needs to be a way to include additional paths in this patch. One important question I would have in these cases is if you expect these to be set only on the master. If not, then is this a problem and if not then how do you handle the fail-back problem etc?
This also brings up a fairly major concern more generally about control by the way. A lot of cases where pg_rewind is called, the user doesn't necessarily have much control on how it is called. Moreover in many of these cases, the user is probably not in a position to understand the internals well enough to grasp what to check after.
> The problem with an exclude list is that we cannot safely exclude
> configuration files or logs (because these could be named anything), right?
You have the exact same problem with base backups now, and any
configuration files created by extensions are a per-case problem...
Right, but there is a use case difference between "I am taking a backup of a server" and "I need to get the server into rejoin the replication as a standby."
A really good example of where this is a big problem is with replication slots. On a backup I would expect you want replication slots to be copied in. However when setting yourself up as a slave you most certainly do not want to just copy these over from the master unless you have infinite disk space. I would argue that under *no* circumstance should pg_rewind *ever* copy replication slots. But pg_base_backup really *should* (and when provisioning a new slave you should clear these as soon as it is set up).
The pattern that base backups now use is an exclude list. In the
future I would rather see base backups and pg_rewind using the same
infrastructure for both things:
- pg_rewind should use the replication protocol with BASE_BACKUP.
Having it rely on root access now is crazy.
- BASE_BACKUP should have an option where it is possible to exclude
custom paths.
What you are proposing here would make both diverge more, which is in
my opinion not a good approach.
How does rep mgr or other programs using pg_rewind know what to exclude?
Again I am not convinced setting up a replica and taking a backup for disaster recovery are the same use case or have the same requirements.
--
Michael
Best Regards,
Chris Travers
Database Administrator
Saarbrücker Straße 37a, 10405 Berlin
pgsql-hackers by date: