Re: Teach pg_receivewal to use lz4 compression - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Teach pg_receivewal to use lz4 compression
Date
Msg-id YYOYIYvudT0uqmiF@paquier.xyz
Whole thread Raw
In response to Re: Teach pg_receivewal to use lz4 compression  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Teach pg_receivewal to use lz4 compression
Re: Teach pg_receivewal to use lz4 compression
List pgsql-hackers
On Thu, Nov 04, 2021 at 04:31:48PM +0900, Michael Paquier wrote:
> I would have expected read_size to be (readend - readp) to match with
> the remaining data in the read buffer that we still need to read.
> Shouldn't out_size also be LZ4_CHUNK_SZ to match with the size of the
> output buffer where all the contents are read?  By setting it to 1, I
> think that this is doing more loops into LZ4F_decompress() than really
> necessary.  It would not hurt either to memset(0) those buffers before
> they are used, IMO.  I am not completely sure either, but should we
> use the number of bytes returned by LZ4F_decompress() as a hint for
> the follow-up loops?
>
> +#ifdef HAVE_LIBLZ4
> +    while (readp < readend)
> +    {
> +        size_t        read_size = 1;
> +        size_t        out_size = 1;
> +
> +        status = LZ4F_decompress(ctx, outbuf, &out_size,
> +                                 readbuf, &read_size, NULL);

And...  It happens that the error from v9 is here, where we need to
read the amount of remaining data from "readp", and not "readbuf" :)

It is already late here, I'll continue on this stuff tomorrow, but
this looks rather committable overall.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Data is copied twice when specifying both child and parent table in publication
Next
From: Takashi Menjo
Date:
Subject: Re: Map WAL segment files on PMEM as WAL buffers