Re: Extending amcheck to check toast size and compression - Mailing list pgsql-hackers

From Mark Dilger
Subject Re: Extending amcheck to check toast size and compression
Date
Msg-id 29751EFD-91D8-4F2F-8416-030AD067F260@enterprisedb.com
Whole thread Raw
In response to Re: Extending amcheck to check toast size and compression  (Heikki Linnakangas <hlinnaka@iki.fi>)
Responses Re: Extending amcheck to check toast size and compression  (Mark Dilger <mark.dilger@enterprisedb.com>)
Re: Extending amcheck to check toast size and compression  (Greg Stark <stark@mit.edu>)
List pgsql-hackers

> On Jul 14, 2021, at 3:33 AM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
>> +/* The largest valid toast va_rawsize */
>> +#define VARLENA_SIZE_LIMIT 0x3FFFFFFF
>> +
>
> Hmm, a toasted datum cannot be larger than MaxAllocSize, because it's reconstituted in a palloc'd datum, right?

No datum size exceeds MaxAllocSize, and no datum expands when compressed (because for those that do expand under any
particularcompression algorithm, we opt to instead store the datum uncompressed), so no valid toast pointer should
containa va_rawsize field greater than MaxAllocSize.  Any toast pointers that have larger va_rawsize fields are
thereforecorrupt. 

VARLENA_SIZE_LIMIT is defined here equal to MaxAllocSize:

    src/include/utils/memutils.h:#define MaxAllocSize       ((Size) 0x3fffffff) /* 1 gigabyte - 1 */

Earlier versions of the patch used MaxAllocSize rather than defining VARLENA_SIZE_LIMIT, but review comments suggested
thatwas less clear. 

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company






pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: row filtering for logical replication
Next
From: Heikki Linnakangas
Date:
Subject: Re: Polyphase merge is obsolete