Re: pg_basebackup failed to back up large file - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: pg_basebackup failed to back up large file
Date
Msg-id CABUevEx_benGF8W0s==S6zNvy5x_U4gDpeyST-Q4UZJVYr12cA@mail.gmail.com
Whole thread Raw
In response to Re: pg_basebackup failed to back up large file  (Andres Freund <andres@2ndquadrant.com>)
Responses Re: pg_basebackup failed to back up large file  (Andres Freund <andres@2ndquadrant.com>)
Re: pg_basebackup failed to back up large file  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
<p dir="ltr"><br /> On Jun 3, 2014 6:17 PM, "Andres Freund" <<a
href="mailto:andres@2ndquadrant.com">andres@2ndquadrant.com</a>>wrote:<br /> ><br /> > On 2014-06-03 17:57:52
+0200,Magnus Hagander wrote:<br /> > > On Tue, Jun 3, 2014 at 5:42 PM, Tom Lane <<a
href="mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>>wrote:<br /> > > > What we had better do, IMO, is fix
thingsso that we don't have a filesize<br /> > > > limit in the basebackup format.  After a bit of googling, I
foundout that<br /> > > > recent POSIX specs for tar format include "extended headers" that among<br /> >
>> other things support member files of unlimited size [1].  Rather than<br /> > > > fooling with
partialfixes, we should make the basebackup logic use an<br /> > > > extended header when the file size is
overINT_MAX.<br /> ><br /> > > Yeah, pax seems to be the way to go. It's at least supported by GNU tar -<br />
>> is it also supported on say BSD, or other popular platforms? (The size<br /> > > extension in the
generalustar format seems to be, so it would be a shame<br /> > > if this one is less portable)<br /> ><br />
>PG's tar.c already uses the ustar format and the referenced extension is<br /> > an extension to ustar as far as
Iunderstand it. So at least tarballs<br /> > with files < 8GB would still continue to be readable with all
currently<br/> > working implementations. <p dir="ltr">Yeah, that is a clear advantage of that method. Didn't read
upon pax format backwards compatibility, does it have some trick to achieve something similar? <p dir="ltr">/Magnus  

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: pg_basebackup failed to back up large file
Next
From: Andres Freund
Date:
Subject: Re: pg_basebackup failed to back up large file