Re: [HACKERS] Custom compression methods (mac+lz4.h) - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [HACKERS] Custom compression methods (mac+lz4.h)
Date
Msg-id 504396.1616368310@sss.pgh.pa.us
Whole thread Raw
In response to Re: [HACKERS] Custom compression methods (mac+lz4.h)  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: [HACKERS] Custom compression methods (mac+lz4.h)  (Justin Pryzby <pryzby@telsasoft.com>)
List pgsql-hackers
I wrote:
> Justin Pryzby <pryzby@telsasoft.com> writes:
>> On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
>>> This seems somewhat repeatable (three identical failures in three
>>> attempts).  Not sure why I did not see it yesterday; but anyway,
>>> there is something wrong with partial detoasting for LZ4.

>> With what version of LZ4 ?

> RHEL8's, which is
> lz4-1.8.3-2.el8.x86_64

I hate to be the bearer of bad news, but this suggests that
LZ4_decompress_safe_partial is seriously broken in 1.9.2
as well:

https://github.com/lz4/lz4/issues/783

Maybe we cannot rely on that function for a few more years yet.

Also, I don't really understand why this code:

    /* slice decompression not supported prior to 1.8.3 */
    if (LZ4_versionNumber() < 10803)
        return lz4_decompress_datum(value);

It seems likely to me that we'd get a flat out build failure
from library versions lacking LZ4_decompress_safe_partial,
and thus that this run-time test is dead code and we should
better be using a configure probe if we intend to allow old
liblz4 versions.  Though that might be moot.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: replication cleanup code incorrect way to use of HTAB HASH_REMOVE ?
Next
From: Andres Freund
Date:
Subject: Re: Table AM and DDLs