[HACKERS] pg_rewind proposed scope and interface changes - Mailing list pgsql-hackers

From Chris Travers
Subject [HACKERS] pg_rewind proposed scope and interface changes
Date
Msg-id CAN-RpxCQyPBCp+86o0Eh4DLjzfPTug7P1ddd0crsTue=iKumpg@mail.gmail.com
Whole thread Raw
Responses Re: [HACKERS] pg_rewind proposed scope and interface changes
List pgsql-hackers

Hi all;


After working with pg_rewind a bit more it has become clear to me that some aspects of the ideal scope of the program are a bit unclear.  This is in part because the majority of cases where you use the utility, a small amount of data loss is accepted due to the tradeoff of continuity.


Basic Responsibility:


pg_rewind’s core responsibility is to restore data directories so that they are compatible with following another server. 


Expanded Options:

As with pg_basebackup, wal files and related data may need to be specified separately.


Pre-run checks:


PG_VERSION must be the same on target and source or else we stop.


We then check the timelines and if they have not diverged don’t do anything.


Processed directories:


pg_basebackup must *always* process global, base, and pg_tablespace directories regardless of any WAL


Additionally the wal, xact, timestamp and logical directories must be processed in some way.


  * if --wal=sync the directories are processed the way they are today

  * if --wal=clear then the contents of the directories are cleared and replication is assumed to be used to bring the system up after.  Note this will need to come with warning about the need for replication slots.


No other files will be clobbered.


This means that the basic processing file list will look like this:


Base, global, pg_tablespace


With

pg_wal, pg_xact, pg_commit_ts, pg_logical added if wal strategy is set to sync.


Proposed patch roadmap:

  1. Improve WAL handling by allowing unlinking of all regular files in those directories during the rewind process.  Make —wal=sync the default for now because that is backwards compatible.  Longer-run we may consider switching the default.
  2. Restrict main operation to the directories listed above.
Problems that we will not try to solve:

     * Rewinding past table creation orphans table file.  This is a complex topic on its own and probably needs a separate utility.

Thoughts?

--
Best Regards,
Chris Travers
Database Administrator

Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com 
Saarbrücker Straße 37a, 10405 Berlin

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] Constraint exclusion for partitioned tables
Next
From: Ildus Kurbangaliev
Date:
Subject: Re: [HACKERS] Custom compression methods