pgsql: Add support for LZ4 with compression of full-page writes in WAL - Mailing list pgsql-committers

From Michael Paquier
Subject pgsql: Add support for LZ4 with compression of full-page writes in WAL
Date
Msg-id E1ly3Lv-0002vi-BO@gemulon.postgresql.org
Whole thread Raw
Responses Re: pgsql: Add support for LZ4 with compression of full-page writes in WAL
List pgsql-committers
Add support for LZ4 with compression of full-page writes in WAL

The logic is implemented so as there can be a choice in the compression
used when building a WAL record, and an extra per-record bit is used to
track down if a block is compressed with PGLZ, LZ4 or nothing.

wal_compression, the existing parameter, is changed to an enum with
support for the following backward-compatible values:
- "off", the default, to not use compression.
- "pglz" or "on", to compress FPWs with PGLZ.
- "lz4", the new mode, to compress FPWs with LZ4.

Benchmarking has showed that LZ4 outclasses easily PGLZ.  ZSTD would be
also an interesting choice, but going just with LZ4 for now makes the
patch minimalistic as toast compression is already able to use LZ4, so
there is no need to worry about any build-related needs for this
implementation.

Author: Andrey Borodin, Justin Pryzby
Reviewed-by: Dilip Kumar, Michael Paquier
Discussion: https://postgr.es/m/3037310D-ECB7-4BF1-AF20-01C10BB33A33@yandex-team.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/4035cd5d4eee4dae797bfc77ab07f8dcd8781b41

Modified Files
--------------
doc/src/sgml/config.sgml                      | 14 +++--
doc/src/sgml/install-windows.sgml             |  2 +-
doc/src/sgml/installation.sgml                |  5 +-
doc/src/sgml/standalone-profile.xsl           |  4 ++
src/backend/access/transam/xlog.c             |  2 +-
src/backend/access/transam/xloginsert.c       | 73 +++++++++++++++++++++++----
src/backend/access/transam/xlogreader.c       | 58 ++++++++++++++++-----
src/backend/utils/misc/guc.c                  | 36 +++++++++----
src/backend/utils/misc/postgresql.conf.sample |  3 +-
src/bin/pg_waldump/pg_waldump.c               | 19 +++++--
src/include/access/xlog.h                     | 10 +++-
src/include/access/xlogrecord.h               | 16 +++---
src/tools/pgindent/typedefs.list              |  1 +
13 files changed, 191 insertions(+), 52 deletions(-)


pgsql-committers by date:

Previous
From: Noah Misch
Date:
Subject: pgsql: Remove XLogFileInit() ability to unlink a pre-existing file.
Next
From: Michael Paquier
Date:
Subject: pgsql: Fix compilation warning in xloginsert.c