Re: [HACKERS] Proposal: pg_rewind to skip config files - Mailing list pgsql-hackers

From Chris Travers
Subject Re: [HACKERS] Proposal: pg_rewind to skip config files
Date
Msg-id CAN-RpxBtBRca+KxeVF286QaBcABJwngf4yE3ZmJ=yoCv8_du3g@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Proposal: pg_rewind to skip config files  (Michael Paquier <michael.paquier@gmail.com>)
Responses Re: [HACKERS] Proposal: pg_rewind to skip config files
List pgsql-hackers


On Tue, Sep 5, 2017 at 1:04 PM, Michael Paquier <michael.paquier@gmail.com> wrote:
On Tue, Sep 5, 2017 at 7:54 PM, Vladimir Borodin <root@simply.name> wrote:
> 5 сент. 2017 г., в 12:31, Chris Travers <chris.travers@adjust.com>
> написал(а):
>
> I think the simplest solution for now is to skip any files ending in .conf,
> .log, and serverlog.

This is not a portable solution. Users can include configuration files
with the names they want. So the current patch as proposed is
definitely not something worth it.

Actually that is exactly why I think the long-term solution is to figure out what we need to copy and not copy anything we don't recognise.

That means the following directories as far as I can see:
 * base
 * global
 * pg_xlog/pg_wal
 * pg_clog/pg_xact
 * pg_commit_ts
 * pg_twophase
 * pg_snapshots?

Are there any other directories I am missing?


At any rate, I think the current state makes it very difficult to test rewind adequately, and it makes it extremely difficult to use in a non-trivial environment because you have to handle replication slots, configuration files, and so forth yourself, and you have to be aware that these *may* or *may not* be consistently clobbered by a rewind, so you have to have some way of applying another set of files in following a rewind.

If nothing else we ought to *at least* special case the recovery.conf and the postgresql.auto.conf, and pg_replslot because these are always located there and should never be clobbered.
 

> For example, in archive_command we put WALs for archiving from
> pg_xlog/pg_wal into another directory inside PGDATA and than another cron
> task makes real archiving. This directory ideally should be skipped by
> pg_rewind, but it would not be handled by proposed change.

I would be curious to follow the reasoning for such a two-phase
archiving (You basically want to push it in two places, no? But why
not just use pg_receivexlog then?). This is complicated to handle from
the point of view of availability and backup reliability + durability.

> While it is definitely an awful idea the user can easily put something
> strange (i.e. logs) to any important directory in PGDATA (i.e. into base or
> pg_wal). Or how for example pg_replslot should be handled (I asked about it
> a couple of years ago [1])? It seems that a glob/regexp for things to skip
> is a more universal solution.
>
> [1]
> https://www.postgresql.org/message-id/flat/8DDCCC9D-450D-4CA2-8CF6-40B382F1F699%40simply.name

Well, keeping the code simple is not always a bad thing. Logs are an
example that can be easily countered, as well as archives in your
case.


 
--
Michael



--
Best Regards,
Chris Travers
Database Administrator

Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com 
Saarbrücker Straße 37a, 10405 Berlin

pgsql-hackers by date:

Previous
From: Vladimir Borodin
Date:
Subject: Re: [HACKERS] Proposal: pg_rewind to skip config files
Next
From: Peter Eisentraut
Date:
Subject: Re: [HACKERS] Disallowing multiple queries per PQexec()