Re: Checking for missing heap/index files - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: Checking for missing heap/index files
Date
Msg-id 20220617155633.ikohjcc3ov4ya6tl@alvherre.pgsql
Whole thread Raw
In response to Re: Checking for missing heap/index files  (Stephen Frost <sfrost@snowman.net>)
Responses Re: Checking for missing heap/index files
List pgsql-hackers
On 2022-Jun-09, Stephen Frost wrote:

> TL;DR: if you're removing files from a directory that you've got an
> active readdir() running through, you might not actually get all of the
> *existing* files.  Given that PG is happy to remove files from PGDATA
> while a backup is running, in theory this could lead to a backup utility
> like pgbackrest or pg_basebackup not actually backing up all the files.
> 
> Now, pgbackrest runs the readdir() very quickly to build a manifest of
> all of the files to backup, minimizing the window for this to possibly
> happen, but pg_basebackup keeps a readdir() open during the entire
> backup, making this more possible.

Hmm, this sounds pretty bad, and I agree that a workaround should be put
in place.  But where is pg_basebackup looping around readdir()?  I
couldn't find it.  There's a call to readdir() in FindStreamingStart(),
but that doesn't seem to match what you describe.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/



pgsql-hackers by date:

Previous
From: Aleksander Alekseev
Date:
Subject: Re: [PATCH] Compression dictionaries for JSONB
Next
From: Bruce Momjian
Date:
Subject: Re: Typo in ro.po file?