Re: backup manifests - Mailing list pgsql-hackers

Hi,

On 2020-03-27 15:29:02 -0400, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 11:26 AM Stephen Frost <sfrost@snowman.net> wrote:
> > > Seems better to (later?) add support for generating manifests for WAL
> > > files, and then have a tool that can verify all the manifests required
> > > to restore a base backup.
> >
> > I'm not trying to expand on the feature set here or move the goalposts
> > way down the road, which is what seems to be what's being suggested
> > here.  To be clear, I don't have any objection to adding a generic tool
> > for validating WAL as you're talking about here, but I also don't think
> > that's required for pg_validatebackup.  What I do think we need is a
> > check of the WAL that's fetched when people use pg_basebackup -Xstream
> > or -Xfetch.  pg_basebackup itself has that check because it's critical
> > to the backup being successful and valid.  Not having that basic
> > validation of a backup really just isn't ok- there's a reason
> > pg_basebackup has that check.
> 
> I don't understand how this could be done without significantly
> complicating the architecture. As I said before, -Xstream sends WAL
> over a separate connection that is unrelated to the one running
> BASE_BACKUP, so the base-backup connection doesn't know what to
> include in the manifest. Now you could do something like: once all of
> the WAL files have been fetched, the client checksums all of those and
> sends their names and checksums to the server, which turns around and
> puts them into the manifest, which it then sends back to the client.
> But that is actually quite a bit of additional complexity, and it's
> pretty strange, too, because now you have the client checksumming some
> files and the server checksumming others. I know you mentioned a few
> different ideas before, but I think they all kinda have some problem
> along these lines.

How about having separate manifests for segments? And have them stay
separate? And then have an option to verify the manifests for all the
WAL files that are required for a specific restore? The easiest way
would be to just add a separate manifest file for each segment, and name
them accordingly. But inventing a naming pattern that specifies both
start-end segments wouldn't be hard either, and result in fewer
manifests.

Base backups (in the backup sense, not for bringing up replicas etc)
without the ability to apply newer WAL are fairly pointless imo. And if
newer WAL is applied, there's not much point in just verifying the WAL
that's necessary to restore the base backup. Instead you'd want to be
able to verify all the WAL since the base backup to the "current" point
(or the next base backup).

For me having something inside pg_basebackup (or the server, for
-Xfetch) that somehow includes the WAL files in the manifest doesn't
really gain us much - it's obviously not something that'll help us to
verify all the WAL that needs to be applied (to either get the base
backup into a consistent state, or to roll forward to the desired
point).



> I also kinda disagree with the idea that the WAL should be considered
> an integral part of the backup. I don't know how pgbackrest does
> things, but BART stores each backup in a separate directly without any
> associated WAL, and then keeps all the WAL together in a different
> directory. I imagine that people who are using continuous archiving
> also tend to use -Xnone, or if they do backups by copying the files
> rather than using pg_backrest, they exclude pg_wal. In fact, for
> people with big, important databases, I'd assume that would be the
> normal pattern. You presumably wouldn't want to keep one copy of the
> WAL files taken during the backup with the backup itself, and a
> separate copy in the archive.

+1

I also don't see them as being as important, due to the already existing
checksums (which are of a much much much higher quality than what we
have for database pages, both by being wider, and by being much more
frequent in most cases). There's obviously a need to validate the WAL in
a nicer way than scripting pg_waldump - but that seems separate anyway.

Greetings,

Andres Freund