Re: [HACKERS] Custom compression methods (mac+lz4.h) - Mailing list pgsql-hackers

Justin Pryzby <pryzby@telsasoft.com> writes:
> On Sun, Mar 21, 2021 at 07:11:50PM -0400, Tom Lane wrote:
>> I hate to be the bearer of bad news, but this suggests that
>> LZ4_decompress_safe_partial is seriously broken in 1.9.2
>> as well:
>> https://github.com/lz4/lz4/issues/783

> Ouch

Actually, after reading that closer, the problem only affects the
case where the compressed-data-length passed to the function is
a lie.  So it shouldn't be a problem for our usage.

Also, after studying the documentation for LZ4_decompress_safe
and LZ4_decompress_safe_partial, I realized that liblz4 is also
counting on the *output* buffer size to not be a lie.  So we
cannot pass it a number larger than the chunk's true decompressed
size.  The attached patch resolves the issue I'm seeing.

            regards, tom lane
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..74e449992a 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -220,6 +220,10 @@ lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
     if (LZ4_versionNumber() < 10803)
         return lz4_decompress_datum(value);

+    /* liblz4 assumes that slicelength is not an overestimate */
+    if (slicelength >= VARRAWSIZE_4B_C(value))
+        return lz4_decompress_datum(value);
+
     /* allocate memory for the uncompressed data */
     result = (struct varlena *) palloc(slicelength + VARHDRSZ);