Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC) - Mailing list pgsql-hackers

From Kevin Grittner
Subject Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)
Date
Msg-id 46FCCB89.EE98.0025.0@wicourts.gov
Whole thread Raw
In response to Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)  ("Zeugswetter Andreas ADI SD" <Andreas.Zeugswetter@s-itsolutions.at>)
Responses Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)
List pgsql-hackers
>>> On Fri, Sep 28, 2007 at  5:53 AM, in message
<E1539E0ED7043848906A8FF995BDA57902685F43@m0143.s-mxs.net>, "Zeugswetter
Andreas ADI SD" <Andreas.Zeugswetter@s-itsolutions.at> wrote:
> I think you misunderstood what I meant.
> The actual archive command is constructed by expanding certain
> placeholders.
> I am suggesting to add such a placeholder for the size of the filled
> part of the log.
>
> A hypothetical example (note suggested %b placeholder for size in
> bytes):
> archive_command=dd if=%p of=/backup/WAL/%f bs=1 count=%b
>
> This allows to avoid unnecessary io for the backup of partially filled
> logs.
I did understand what you were suggesting regarding the size placeholder.
What didn't click is that the filter might not be necessary at all if we
had that.  Thanks for clarifying that with an example.
Are you also suggesting that any code which depends on the log segment
files being at the full size should be changed, too?  If not, I think
your example would need to cat the dd you showed with one which drew
from /dev/zero.  I'll run a few tests with full and nearly empty files
using hand-generated values and see how the performance of this in
front of gzip compares to the filter.
-Kevin




pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)
Next
From: Zdenek Kotala
Date:
Subject: Re: [pgsql-packagers] Getting to 8.3 beta1