Re: Fix detection of truncated zstd-compressed backups - Mailing list pgsql-hackers

From Zsolt Parragi
Subject Re: Fix detection of truncated zstd-compressed backups
Date
Msg-id CAN4CZFPbcBD+5Xj8XrDa1KQoV75-aWBiKs8n-zEJmQnBG4D3KA@mail.gmail.com
Whole thread
In response to Re: Fix detection of truncated zstd-compressed backups  (Chao Li <li.evan.chao@gmail.com>)
List pgsql-hackers
Hello

An automatic post-commit claude review pointed out that this broke
restoring directory format dumps with an empty large object, when
compressed with zstd or lz4:

    SELECT lo_create(0);

    $ pg_dump -Fd -Z zstd -f dump postgres
    $ pg_restore -f /dev/null dump
    pg_restore: error: could not decompress data: compressed stream is
incomplete

    $ pg_dump -Fd -Z lz4 -f dump postgres
    $ pg_restore -f /dev/null dump
    pg_restore: error: could not read from input file: Input/output error
    pg_restore: error: could not read from input file: Input/output error

The compressed file handles only start a frame on the first write, so
the blob_NNN.dat.zst/.lz4 file of an empty large object is 0 bytes,
and the new check treats that as a truncated stream. Custom format and
gzip are fine.

The attached patch makes the reader accept a file with no input at all
as a complete stream, a stream that started a frame and then stopped
is still rejected.

Attachment

pgsql-hackers by date:

Previous
From: "ZizhuanLiu X-MAN"
Date:
Subject: Re: examine_variable ignored CollateExpr
Next
From: Srinath Reddy Sadipiralla
Date:
Subject: Fwd: [RFC PATCH v1] On-demand WAL replay: accept connections before crash recovery has applied the WAL