Re: Add notes to pg_combinebackup docs - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: Add notes to pg_combinebackup docs
Date
Msg-id cfcf14ea-8400-450f-a7d9-3f8eb0ab8b27@enterprisedb.com
Whole thread Raw
In response to Re: Add notes to pg_combinebackup docs  (David Steele <david@pgmasters.net>)
Responses Re: Add notes to pg_combinebackup docs
List pgsql-hackers
On 4/12/24 11:50, David Steele wrote:
> On 4/12/24 19:09, Magnus Hagander wrote:
>> On Fri, Apr 12, 2024 at 12:14 AM David Steele <david@pgmasters.net
>>
>> ...>>
>>      > But yeah, having to keep the backups as expanded directories is
>> not
>>      > great, I'd love to have .tar. Not necessarily because of the disk
>>     space
>>      > (in my experience the compression in filesystems works quite
>> well for
>>      > this purpose), but mostly because it's more compact and allows
>>     working
>>      > with backups as a single piece of data (e.g. it's much cleared
>>     what the
>>      > checksum of a single .tar is, compared to a directory).
>>
>>     But again, object stores are commonly used for backup these days and
>>     billing is based on data stored rather than any compression that
>> can be
>>     done on the data. Of course, you'd want to store the compressed
>> tars in
>>     the object store, but that does mean storing an expanded copy
>> somewhere
>>     to do pg_combinebackup.
>>
>> Object stores are definitely getting more common. I wish they were
>> getting a lot more common than they actually are, because they
>> simplify a lot.  But they're in my experience still very far from
>> being a majority.
> 
> I see it the other way, especially the last few years. The majority seem
> to be object stores followed up closely by NFS. Directly mounted storage
> on the backup host appears to be rarer.
> 

One thing I'd mention is that not having built-in support for .tar and
.tgz backups does not mean it's impossible to use pg_combinebackup with
archives. You can mount them using e.g. "ratarmount" and then use that
as source directories for pg_combinebackup.

It's not entirely friction-less because AFAICS it's necessary to do the
backup in plain format and then do the .tar to have the expected "flat"
directory structure (and not manifest + 2x tar). But other than that it
seems to work fine (based on my limited testing).


FWIW the "archivemount" performs terribly, so adding this capability
into pg_combinebackup is clearly far from trivial.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: Should we add a compiler warning for large stack frames?
Next
From: Daniel Gustafsson
Date:
Subject: Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?