pgsql: Fix detection of truncated zstd and LZ4 dump data - Mailing list pgsql-committers

From Daniel Gustafsson
Subject pgsql: Fix detection of truncated zstd and LZ4 dump data
Date
Msg-id E1xF90L-00000000wPq-42oQ@gemulon.postgresql.org
Whole thread
List pgsql-committers
Fix detection of truncated zstd and LZ4 dump data

pg_restore did not verify that zstd and LZ4 decompression completed
the final frame when reading custom and directory-format dumps. As a
result, it could accept a dump whose compressed data was truncated.

Track final-frame completion and reject incomplete compressed input.
For zstd, flush any internally buffered output before performing the
final check.

Add regression tests for truncated zstd and LZ4 data in custom and
directory-format dumps and also address a potentially empty error
message while in there.

Author: Chao Li <lic@highgo.com>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/5962B878-C43D-4EBC-9E95-1F945CE5E586@gmail.com
Backpatch-through: 19

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e01fc6315f19a794301ca8889d37ae629862e2ac

Modified Files
--------------
src/bin/pg_dump/compress_gzip.c           |   6 +-
src/bin/pg_dump/compress_lz4.c            |  30 ++++++-
src/bin/pg_dump/compress_zstd.c           |  67 ++++++++++++++-
src/bin/pg_dump/t/006_pg_dump_compress.pl | 136 ++++++++++++++++++++++++++++++
4 files changed, 233 insertions(+), 6 deletions(-)


pgsql-committers by date:

Previous
From: Michael Paquier
Date:
Subject: pgsql: Treat any negative ws_file as closed in xlogreader.c
Next
From: Daniel Gustafsson
Date:
Subject: pgsql: Fix detection of truncated zstd and LZ4 dump data