Re: Teach pg_receivewal to use lz4 compression - Mailing list pgsql-hackers

From Dilip Kumar
Subject Re: Teach pg_receivewal to use lz4 compression
Date
Msg-id CAFiTN-urjKKCwu+YA5VYLG40_RLst-B=BxZNKXGpuTgh8hoFtA@mail.gmail.com
Whole thread Raw
In response to Teach pg_receivewal to use lz4 compression  (gkokolatos@pm.me)
Responses Re: Teach pg_receivewal to use lz4 compression
List pgsql-hackers
On Tue, Jun 29, 2021 at 8:15 PM <gkokolatos@pm.me> wrote:
>
> Hi,
>
> The program pg_receivewal can use gzip compression to store the received WAL.
> This patch teaches it to be able to use lz4 compression if the binary is build
> using the -llz4 flag.

+1 for the idea

Some comments/suggestions on the patch

1.
@@ -90,7 +91,8 @@ usage(void)
  printf(_("      --synchronous      flush write-ahead log immediately
after writing\n"));
  printf(_("  -v, --verbose          output verbose messages\n"));
  printf(_("  -V, --version          output version information, then exit\n"));
- printf(_("  -Z, --compress=0-9     compress logs with given
compression level\n"));
+ printf(_("  -I, --compress-program use this program for compression\n"));

Wouldn't it be better to call it compression method instead of
compression program?

2.
+ printf(_("  -Z, --compress=0-9     compress logs with given
compression level (available only with compress-program=zlib)\n"));

I think we can somehow use "acceleration" parameter of lz4 compression
to map on compression level, It is not direct mapping but
can't we create some internal mapping instead of completely ignoring
this option for lz4, or we can provide another option for lz4?

3. Should we also support LZ4 compression using dictionary?

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619
Next
From: Antonin Houska
Date:
Subject: Re: POC: Cleaning up orphaned files using undo logs