Re: Compressed TOAST Slicing - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: Compressed TOAST Slicing
Date
Msg-id 8473F4DC-D250-4C9E-B29D-311F75C2D9CC@yandex-team.ru
Whole thread Raw
In response to Re: Compressed TOAST Slicing  (Paul Ramsey <pramsey@cleverelephant.ca>)
List pgsql-hackers

> 12 марта 2019 г., в 19:40, Paul Ramsey <pramsey@cleverelephant.ca> написал(а):
>
>> On Mar 11, 2019, at 10:42 PM, Michael Paquier <michael@paquier.xyz> wrote:
>>
>> int32
>> pglz_decompress(const char *source, int32 slen, char *dest,
>> -                               int32 rawsize)
>> +                               int32 rawsize, bool is_slice)
>
> The sanity check is just that both buffers are completely read they reach their respective ends. With a partial
bufferon one side, that check just will definitionally not  happen when slicing (it’s not possible to know a priori
whatlocation in the compressed buffer corresponds to a location in the uncompressed one). I can ensure the old API
stillholds for pglz_decompress() and add a new pglz_decompress_slice() that takes the parameter, is that sufficient? 

I think that providing two separate entry points for this functionality is better option.
The word "slice" is widely used for [start:end] slicing, not sure it's good word. But I'm not good in English.

Either way we could replace

if (dp != destend || sp != srcend)
    return -1;

with

if (dp != destend && sp != srcend)
    return -1;

and that's it. || defends from data corruption and some kind of programming mistakes, but actually that's not the
purposeof data compression. 

Best regards, Andrey Borodin.

pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: Update does not move row across foreign partitions in v11
Next
From: Andres Freund
Date:
Subject: Re: Compressed TOAST Slicing