Re: WAL "low watermark" during base backup - Mailing list pgsql-hackers

From Florian Pflug
Subject Re: WAL "low watermark" during base backup
Date
Msg-id 5286314D-5005-4A19-BBF0-C0D8D906BDCC@phlo.org
Whole thread Raw
In response to Re: WAL "low watermark" during base backup  (Magnus Hagander <magnus@hagander.net>)
Responses Re: WAL "low watermark" during base backup
List pgsql-hackers
On Sep9, 2011, at 13:48 , Magnus Hagander wrote:
> On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine <dimitri@2ndquadrant.fr> wrote:
>> Magnus Hagander <magnus@hagander.net> writes:
>>>> If you must have this then make pg_basebackup copy xlog files
>>>> regularly during the backup. That way your backup can take forever and
>>>> your primary disk won't fill up. In many cases it actually will take
>>>> forever, but at least we don't take down the primary.
>>>
>>> There is a patch to do something like that as well sitting on the CF
>>> page. I don't believe one necessarily excludes the other.
>>
>> I'm not getting why we need the later one when we have this older one?
>
> One of them is for the simple case. It requires a single connection to
> the server, and it supports things like writing to tarfiles and
> compression.
>
> The other one is more compelx. It uses multiple connections (one for
> the base, one for the xlog), and as such doesn't support writing to
> files, only directories.

I guess the real question is, why can't we stream the WALs as they are
generated instead of at the end even over a single connection and when
writing tarfiles?

Couldn't we send all available WAL after each single data-file instead
of waiting for all data files to be transferred before sending WAL?

best regards,
Florian Pflug



pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: regress test failed
Next
From: Dimitri Fontaine
Date:
Subject: Re: WAL "low watermark" during base backup