Re: Fixing pg_basebackup with tablespaces found in $PGDATA - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: Fixing pg_basebackup with tablespaces found in $PGDATA
Date
Msg-id CABUevExZpOyOEKM4hdz0Z+XYfqdq8h-+HSmVi9RyiUq4ASdc9w@mail.gmail.com
Whole thread Raw
In response to Fixing pg_basebackup with tablespaces found in $PGDATA  (Dimitri Fontaine <dimitri@2ndQuadrant.fr>)
Responses Re: Fixing pg_basebackup with tablespaces found in $PGDATA
List pgsql-hackers
On Wed, Jan 1, 2014 at 11:53 PM, Dimitri Fontaine <dimitri@2ndquadrant.fr> wrote:
Hi,

As much as I've seen people frown upon $subject, it still happens in the
wild, and Magnus seems to agree that the current failure mode of our
pg_basebackup tool when confronted to the situation is a bug.

So here's a fix, attached.

To reproduce, mkdir -p $PGDATA/tbs/foo then CREATE TABLESPACE there, and
then pg_basebackup your server. If doing so from the same server, as I
did, then pick the tar format, as here:

  pg_basebackup -Ft -z -c fast -v -X fetch -D /tmp/backup

Then use tar to see that the base backup contains the whole content of
your foo tablespace, and if you did create another tablespace within
$PGDATA/pg_tblspc (which is the other common way to trigger that issue)
then add it to what you want to see:

  tar tzvf /tmp/backup/base.tar.gz pg_tblspc tbs/foo pg_tblspc/bar

Note that empty directories are expected, so tar should output their
entries. Those directories are where you need to be restoring the
tablespace tarballs.

When using pg_basebackup in plain mode, the error is that you get a copy
of all your tablespaces first, then the main PGDATA is copied over and
as the destination directories already do exists (and not empty) the
whole backup fails there.

The bug should be fixed against all revisions of pg_basebackup, though I
didn't try to apply this very patch on all target branches.

I had a second round of thought about this, and I don't think this one is going to work. Unfortunately, it's part of the advice I gave you yesterday that was bad I think :)

We can't get away with just comparing the relative part of the pathname. Because it will fail if there is another path with exactly the same length, containing the tablespace.

I think we might want to store a value in the tablespaceinfo struct indicating whether it's actually inside PGDATA (since we have the full path at that point), and then skip it based on that instead. Or store and pass the value of getcwd() perhaps.

Or am I thinking wrong now instead? :)

I've attached a slightly updated patch - I changed around a bit of logic order and updated some comments during my review. And added error-checking.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/
Attachment

pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: [PATCH] Remove some duplicate if conditions
Next
From: Rushabh Lathia
Date:
Subject: ERROR: missing chunk number 0 for toast value