Re: pg_waldump: support decoding of WAL inside tarfile - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: pg_waldump: support decoding of WAL inside tarfile
Date
Msg-id CA+hUKG+A-=BKP69ze5oc4O5RSBZtKk7dGoqYLeOUQSo8xugTyg@mail.gmail.com
Whole thread
In response to Re: pg_waldump: support decoding of WAL inside tarfile  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: pg_waldump: support decoding of WAL inside tarfile
List pgsql-hackers
On Fri, Apr 3, 2026 at 7:48 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> *This* tar defaults to:
> --format=posix -f- -b20 --quoting-style=escape --rmt-command=/usr/bin/rmt
> --rsh-command=/usr/bin/ssh

From some quick googling, OpenSUSE is the only distro I could find
that does this, but the fine manual says they plan to make pax the
default upstream so eventually it might be everywhere:

https://www.gnu.org/software/tar/manual/html_section/Formats.html

> Interestingly, pg_verifybackup's t/003_corruption.pl test also fails
> with the same issue, so apparently this platform is even more
> aggressive about sparse-ifying files than Thomas' FreeBSD box.

Looks like sparse files and BTRFS might be a red herring then, if it's
simply been told to put a pax header on every file?

I think my system might be defaulting to "restricted pax", meaning
that pax headers are added to individual files only if necessary,
path-too-long etc, magic sparse format triggered, etc.

https://man.freebsd.org/cgi/man.cgi?query=libarchive-formats

> I'm inclined to push the logic for selecting these tar options
> into some common subroutine in Test::Utils, rather than having
> two copies (and maybe more later).

How about using --format=ustar, instead of that sparse control stuff?
That solves the BSD tar + ZFS problem and the OpenSUSE GNU tar default
format problem in one fell swoop, and actually says what we want
explicitly.

Should we have a <note> near the --path/--wal-path documentation to
state explicitly that usta format is required, and that pax is not
[yet] supported, and that --format=ustar might be required on some
systems?



pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: Small and unlikely overflow hazard in bms_next_member()
Next
From: Tom Lane
Date:
Subject: Re: pg_waldump: support decoding of WAL inside tarfile