Re: Block-level CRC checks - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Block-level CRC checks
Date
Msg-id 5151.1259693915@sss.pgh.pa.us
Whole thread Raw
In response to Re: Block-level CRC checks  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: Block-level CRC checks  (Aidan Van Dyk <aidan@highrise.ca>)
Page-level version upgrade (was: Block-level CRC checks)  (decibel <decibel@decibel.org>)
List pgsql-hackers
Simon Riggs <simon@2ndQuadrant.com> writes:
> On Tue, 2009-12-01 at 13:05 -0500, Bruce Momjian wrote:
>> When we read in a page that has an invalid CRC, we check the page to see
>> which hint bits are _not_ set, and we try setting them to see if can get
>> a matching CRC.

> Perhaps we could store a sector-based parity bit each 512 bytes in the
> block. If there are an even number of hint bits set, if odd we unset the
> parity bit. So whenever we set a hint bit we flip the parity bit for
> that sector. That way we could detect which sectors are potentially
> missing in an effort to minimize the number of combinations we need to
> test.

Actually, the killer problem with *any* scheme involving "guessing"
is that each bit you guess translates directly to removing one bit
of confidence from the CRC value.  If you try to guess at as many
as 32 bits, it is practically guaranteed that you will find a
combination that makes a 32-bit CRC appear to match.  Well before
that, you have degraded the reliability of the error detection to
the point that there's no point.

The bottom line here seems to be that the only practical way to do
anything like this is to move the hint bits into their own area of
the page, and then exclude them from the CRC.  Are we prepared to
once again blow off any hope of in-place update for another release
cycle?
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Block-level CRC checks
Next
From: Tom Lane
Date:
Subject: Re: Deleted WAL files held open by backends in Linux