Re: Allowing multiple concurrent base backups - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: Allowing multiple concurrent base backups
Date
Msg-id AANLkTikpJFtA7645gNCq_232_wu7BHcSmn_QJo56ivpp@mail.gmail.com
Whole thread Raw
In response to Re: Allowing multiple concurrent base backups  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Allowing multiple concurrent base backups  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Tue, Jan 11, 2011 at 19:51, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
>> I implemented this in two ways, and can't decide which I like better:
>
>> 1. The contents of the backup label file are returned to the caller of
>> do_pg_start_backup() as a palloc'd string.
>
>> 2. do_pg_start_backup() creates a temporary file that the backup label
>> is written to (instead of "backup_label").
>
>> Implementation 1 changes more code, as pg_start/stop_backup() need to be
>> changed to write/read from memory instead of file, but the result isn't
>> any more complicated. Nevertheless, I somehow feel more comfortable with 2.
>
> Seems like either one of these is fairly problematic in that you have to
> have some monstrous kluge to get the backup_label file to appear with
> the right name in the tarfile.  How badly do we actually need this?
> I don't think the use-case for concurrent base backups is all that large
> in practice given the I/O hit it's going to involve.

I think it can be done cleaner in the tar file injection - I've been
chatting with Heikki offlist about that. Not sure, but I have a
feeling it does.

As for the use-case, it doesn't necessarily involve a huge I/O hit if
either the entire DB is in RAM (not all that uncommon - though then
the backup finishes quickly as well), or if the *client* is slow, thus
making the server backlogged on sending the base backup.

Having it possible to do it concurrently also makes for *much* easier
use of the feature. It might be just about overlapping with a few
seconds, for example - which would probably have no major effect, but
takes a lot more code on the other end to work around.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Allowing multiple concurrent base backups
Next
From: Tom Lane
Date:
Subject: Re: Bug in pg_describe_object