pgsql: Rework compression options of pg_receivewal - Mailing list pgsql-committers

From Michael Paquier
Subject pgsql: Rework compression options of pg_receivewal
Date
Msg-id E1miSGY-0003uH-PR@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Rework compression options of pg_receivewal

pg_receivewal includes since cada1af the option --compress, to allow the
compression of WAL segments using gzip, with a value of 0 (the default)
meaning that no compression can be used.

This commit introduces a new option, called --compression-method, able
to use as values "none", the default, and "gzip", to make things more
extensible.  The case of --compress=0 becomes fuzzy with this option
layer, so we have made the choice to make pg_receivewal return an error
when using "none" and a non-zero compression level, meaning that the
authorized values of --compress are now [1,9] instead of [0,9].  Not
specifying --compress with "gzip" as compression method makes
pg_receivewal use the default of zlib instead (Z_DEFAULT_COMPRESSION).

The code in charge of finding the streaming start LSN when scanning the
existing archives is refactored and made more extensible.  While on it,
rename "compression" to "compression_level" in walmethods.c, to reduce
the confusion with the introduction of the compression method, even if
the tar method used by pg_basebackup does not rely on the compression
method (yet, at least), but just on the compression level (this area
could be improved more, actually).

This is in preparation for an upcoming patch that adds LZ4 support to
pg_receivewal.

Author: Georgios Kokolatos
Reviewed-by: Michael Paquier, Jian Guo, Magnus Hagander, Dilip Kumar,
Robert Haas
Discussion:
https://postgr.es/m/ZCm1J5vfyQ2E6dYvXz8si39HQ2gwxSZ3IpYaVgYa3lUwY88SLapx9EEnOf5uEwrddhx2twG7zYKjVeuP5MwZXCNPybtsGouDsAD1o2L_I5E=@pm.me

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/d62bcc8b07f921bad105c7a826702c117ea7be58

Modified Files
--------------
doc/src/sgml/ref/pg_receivewal.sgml          |  29 ++++-
src/bin/pg_basebackup/pg_basebackup.c        |   7 +-
src/bin/pg_basebackup/pg_receivewal.c        | 167 +++++++++++++++++++--------
src/bin/pg_basebackup/receivelog.c           |   2 +-
src/bin/pg_basebackup/t/020_pg_receivewal.pl |  16 ++-
src/bin/pg_basebackup/walmethods.c           |  96 ++++++++-------
src/bin/pg_basebackup/walmethods.h           |  16 ++-
src/tools/pgindent/typedefs.list             |   1 +
8 files changed, 231 insertions(+), 103 deletions(-)


pgsql-committers by date:

Previous
From: Peter Geoghegan
Date:
Subject: pgsql: Add another old commit to git-blame-ignore-revs.
Next
From: Michael Paquier
Date:
Subject: pgsql: Fix some thinkos with pg_receivewal --compression-method