Re: pg_basebackup failed to read a file - Mailing list pgsql-general

From Michael Paquier
Subject Re: pg_basebackup failed to read a file
Date
Msg-id 20180816004105.GF3681@paquier.xyz
Whole thread Raw
In response to Re: pg_basebackup failed to read a file  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
On Tue, Aug 14, 2018 at 12:14:59PM -0400, Tom Lane wrote:
> That seems like a pretty expensive thing to do, if there are lots of
> files ... and you'd still end up failing, so it's not moving the ball
> very far.

Yeah, I would think that with many small relations it is going to have a
measurable performance impact if we scan the whole data directory a
second time.

> More generally, this seems closely related to bug #14999 [1]
> which concerned pg_rewind's behavior in the face of unexpected file
> permissions within the data directory.  We ended up not doing anything
> about that except documenting it, which I wasn't very satisfied with,
> but the costs of doing better seemed to exceed the benefits.

Please feel free to read the end of the thread about details on the
matter.  There are many things you could do, all have drawbacks.

> It'd be nice to have a more coherent theory about what needs to be copied
> or not, and not fail on files that could simply be ignored.  Up to now
> we've resisted having any centrally defined knowledge of what can be
> inside a PG data directory, but maybe that bullet needs to be bitten.

Yeah, I have not really come up with a nice idea yet, especially when
things sometimes move with custom files that some users have been
deploying, so I am not completely sure that we'd need to do something
anyway, nor that it is worth the trouble.  One saner strategy may be to
split your custom file into a directory out of the main data folder...
--
Michael

Attachment

pgsql-general by date:

Previous
From: Michael Paquier
Date:
Subject: Re: upgrading from pg 9.3 to 10
Next
From: Laurenz Albe
Date:
Subject: Re: Odd Row Estimates in Query Plan (rows=75)