Re: PATCH: Exclude additional directories in pg_basebackup - Mailing list pgsql-hackers

From Kevin Grittner
Subject Re: PATCH: Exclude additional directories in pg_basebackup
Date
Msg-id CACjxUsOpDM9h-4KkiNJc3qQEKDQVB_cnUc4iv2obUPg7dq2w5Q@mail.gmail.com
Whole thread Raw
In response to Re: PATCH: Exclude additional directories in pg_basebackup  (David Steele <david@pgmasters.net>)
List pgsql-hackers
On Wed, Aug 17, 2016 at 1:50 PM, David Steele <david@pgmasters.net> wrote:

>>> That patch got me thinking about what else could be excluded and after
>>> some investigation I found the following: pg_notify, pg_serial,
>>> pg_snapshots, pg_subtrans.  These directories are all cleaned, zeroed,
>>> or rebuilt on server start.
>>
>> Eh ... I doubt very much that it's safe to blow away the entire
>> contents of an SLRU between shutdown and startup, even if the data is
>> technically transient data that won't be needed again after the system
>> is reset.
>
> I've done pretty extensive testing in pgBackRest and haven't seen issues
> in any supported version (plus I audited each init() function for every
> version back to where it was introduced).  The patch also passes all the
> pg_basebackup TAP tests in master.
>
> If you are correct it may indicate a problem anyway. Consider a standby
> backup where the files in these directories may be incredibly stale
> since they are not replicated.  Once restored to a master should we
> trust anything in these files?
>
> pg_serial, pg_notify, pg_subtrans are not even fsync'd
> (SlruCtl->do_fsync = false).  It's hard to imagine there's anything of
> value in there or that it can be trusted if there is.

The contents of pg_serial could be deleted safely.  As I recall, I
initially had it cleaned out on startup, but Heikki (who was the
main committer for this feature) added SLRU buffer flushing for it
on checkpoint and took out the startup delete code with the
explanation that he thought someone might want to look at the file
contents for debugging purposes.  I would be a bit surprised if
anyone ever has used it for that, but that is the reason the
contents are not deleted just like pg_snapshot and pg_dynshmem.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Bug in intarray bench script
Next
From: Robert Haas
Date:
Subject: Re: Declarative partitioning - another take