Re: Block-level CRC checks - Mailing list pgsql-hackers

On Wed, Dec 2, 2009 at 12:03 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Greg Stark <gsstark@mit.edu> writes:
>> On Tue, Dec 1, 2009 at 10:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> I don't think "relatively cheap" is the right criterion here --- the
>>> question to me is how many assumptions are you making in order to
>>> compute the page's CRC.  Each assumption degrades the reliability
>>> of the check, not to mention creating another maintenance hazard.
>
>> Well the only assumption here is that we know where the line pointers
>> start and end.
>
> ... and what they contain.  To CRC a subset of the page at all, we have
> to put some amount of faith into the page header's pointers.  We can do
> weak checks on those, but only weak ones.  If we process different parts
> of the page differently, we're increasing our trust in those pointers
> and reducing the quality of the CRC check.

I'm not sure we're on the same page.  As I understand it there are
three proposals on the table now:

1) set aside a section of the page to contain only non-checksummed
hint bits. That section has to be relocatable so the crc check would
have to read the start and end address of it from the page header.

2) store the hint bits in the line pointers and skip checking the line
pointers. In that case the crc check would skip any bytes between the
start of the line pointer array and pd_lower (or pd_upper? no point in
crc checking unused bytes is there?)

3) store the hint bits in the line pointers and apply a mask which
masks out the 4 hint bits in each 32-bit word in the region between
the start of the line pointers and pd_lower (or pd_upper again)

These three options all seem to have the same level of interdependence
for the crc check, namely they all depend one or two values in the
page header to specify a range of bytes in the block. None of them
depend on the contents of the line pointers themselves being correct,
only the one or two fields in the header specifying which range of
bytes the hint bits lie in.

For what it's worth I don't think "decreasing the quality of the crc
check" is actually valid. The bottom line is that in all of the above
options if any pointer is invalid we'll be CRCing a different set of
data from the set that originally went into calculating the stored CRC
so we'll be effectively computing a random value which will have a
1/2^32 chance of being the value stored in the CRC field regardless of
anything else.

--
greg