Re: base backup vs. concurrent truncation - Mailing list pgsql-hackers

From Greg Stark
Subject Re: base backup vs. concurrent truncation
Date
Msg-id CAM-w4HMDSTsuVbv3LTeCkgzWawiWD7Y6FVDnefhHs7wmGgO3PQ@mail.gmail.com
Whole thread Raw
In response to Re: base backup vs. concurrent truncation  (Andres Freund <andres@anarazel.de>)
Responses Re: base backup vs. concurrent truncation
List pgsql-hackers
Including the pre-truncation length in the wal record is the obviously
solid approach and I none of the below is a good substitution for it.
But....

On Tue, 25 Apr 2023 at 13:30, Andres Freund <andres@anarazel.de> wrote:
>
> It isn't - but the alternatives aren't great either. It's not that easy to hit
> this scenario, so I think something along these lines is more palatable than
> adding a pass through the entire data directory.

Doing one pass through the entire data directory on startup before
deciding the directory is consistent doesn't sound like a crazy idea.
It's pretty easy to imagine bugs in backup software that leave out
files in the middle of tables -- some of us don't even have to
imagine...

Similarly checking for a stray next segment whenever extending a file
to maximum segment size seems like a reasonable thing to check for
too.

These kinds of checks are the kind of paranoia that catches filesystem
bugs, backup software bugs, cron jobs, etc that we don't even know to
watch for.

-- 
greg



pgsql-hackers by date:

Previous
From: Jehan-Guillaume de Rorthais
Date:
Subject: Re: Unlinking Parallel Hash Join inner batch files sooner
Next
From: David Rowley
Date:
Subject: Re: benchmark results comparing versions 15.2 and 16