Re: [HACKERS] pg_rewind proposed scope and interface changes - Mailing list pgsql-hackers
| From | Chris Travers |
|---|---|
| Subject | Re: [HACKERS] pg_rewind proposed scope and interface changes |
| Date | |
| Msg-id | CAN-RpxA6d6SuT0FdCLwQG1PTW_2CcWC1rM46_G3AkOSSTLqzGQ@mail.gmail.com Whole thread Raw |
| In response to | Re: [HACKERS] pg_rewind proposed scope and interface changes (Michael Paquier <michael.paquier@gmail.com>) |
| List | pgsql-hackers |
On Wed, Sep 13, 2017 at 6:28 AM, Michael Paquier <michael.paquier@gmail.com> wrote:
On Tue, Sep 12, 2017 at 11:52 PM, Chris Travers
<chris.travers@adjust.com> wrote:
> Additionally the wal, xact, timestamp and logical directories must be
> processed in some way.
To what does the term "logical directories" refer to?
> * if --wal=sync the directories are processed the way they are today
> * if --wal=clear then the contents of the directories are cleared and
> replication is assumed to be used to bring the system up after. Note this
> will need to come with warning about the need for replication slots.
Hm. I am not sure in what --wal=clear is helpful. Keeping around WAL
segments from the point of the last checkpoint where WAL forked up to
the point where WAL has forked is helpful, because you don't need to
copy again those WAL segments, be they come from an archive or from
streaming. Copying a set of WAL segments during the rewind of the new
timeline is helpful as well because you don't need to do the copy
again. One configuration where this is helpful is that there is
already an archive local to the target server available with the
segments of the new timeline available.
so maybe clear should just clear diverged wal files. That makes some sense.
> Base, global, pg_tablespace
>
> With
> pg_wal, pg_xact, pg_commit_ts, pg_logical added if wal strategy is set to
> sync.
Skipping some directories in a way similar to what a base backup does
would be nicer I think. We already have a list of those in
basebackup.c in the shape of excludeDirContents and excludeFiles. I
think that it would be a good idea to export those into a header that
pg_rewind could include, and refer to in order to exclude them when
fetching a set of files. At the end of the day, a rewind is a kind of
base backup in itself, and this patch would already serve well a lot
of people.
I think there are several reasons not to just do this based on base backup. For base backup there may be some things we restore as such that we don't want to restore with pg_rewind. Configuration files, logs, and replication slots are the ones that immediately come to mind. Given that configuration files can have arbitrary names and locations, expecting third party failover tooling like repmgr or patroni to pick up on such policies strikes me as asking a bit much. I think we are better off with a program that solves one problem well and solves it right and the problem is rewinding a system so that it can follow a replica that was previously following it.
Having on top of that a way to exclude a wanted set of files and the
log directory (for example: should we look at log_directory and
exclude it from the fetched paths if it is not an absolute path?),
which is smart enough to take care of not removing paths critical for
a rewind like anything in base/, then you are good to go with a
full-blown tool that I think would serve the purposes you are looking
for.
If you are going to go this route, I think you need to look at the config files themselves, parse them, and possibly look at the command line by which PostgreSQL is started. Otherwise I don't know how you expect this to be configured....
--
Michael
Best Regards,
Chris Travers
Database Administrator
Saarbrücker Straße 37a, 10405 Berlin
pgsql-hackers by date: