Re: Block-level CRC checks - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: Block-level CRC checks
Date
Msg-id 200912011805.nB1I5lo02767@momjian.us
Whole thread Raw
In response to Re: Block-level CRC checks  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: Block-level CRC checks  (Simon Riggs <simon@2ndQuadrant.com>)
Re: Block-level CRC checks  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Simon Riggs wrote:
> Also, we might
> 
> * Put all hint bits in the block header to allow them to be excluded
> more easily from CRC checking. If we used 3 more bits from
> ItemIdData.lp_len (limiting tuple length to 4096) then we could store
> some hints in the item pointer. HEAP_XMIN_INVALID can be stored as
> LP_DEAD, since that will happen very quickly anyway. 

OK, here is another idea, maybe crazy:

When we read in a page that has an invalid CRC, we check the page to see
which hint bits are _not_ set, and we try setting them to see if can get
a matching CRC.  If there no missing hint bits and the CRC doesn't
match, we know the page is corrupt.  If two hint bits are missing, we
can try setting one and both of them and see if can get a matching CRC. 
If we can, the page is OK, if not, it is corrupt.

Now if 32 hint bits are missing, but could be based on transaction
status, then we would need 2^32 possible hint bit combinations, so we
can't do the test and we just assume the page is valid.

I have no idea what percentage of corruption this would detect, but it
might have minimal overhead because the overhead only happens when we
detect a non-matching CRC due to a crash of some sort.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


pgsql-hackers by date:

Previous
From: "Joshua D. Drake"
Date:
Subject: Re: Block-level CRC checks
Next
From: "Kevin Grittner"
Date:
Subject: Re: Deleted WAL files held open by backends in Linux