Re: Block-level CRC checks - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Block-level CRC checks
Date
Msg-id 1259678598.13774.13487.camel@ebony
Whole thread Raw
In response to Re: Block-level CRC checks  (Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>)
Responses Re: Block-level CRC checks  (Florian Weimer <fweimer@bfk.de>)
Re: Block-level CRC checks  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers
On Tue, 2009-12-01 at 15:30 +0200, Heikki Linnakangas wrote:
> Bruce Momjian wrote:
> > What might be interesting is to report CRC mismatches if the database
> > was shut down cleanly previously;  I think in those cases we shouldn't
> > have torn pages.
> 
> Unfortunately that's not true. You can crash, leading to a torn page,
> and then start up the database and shut it down cleanly. The torn page
> is still there, even though the last shutdown was a clean one.

There seems to be two ways forwards: journalling or fsck.

We can either

* WAL-log all changes to a page (journalling) (8-byte overhead)

* After a crash disable CRC checks until a full database scan has either
re-checked CRC or found CRC mismatch, report it in the LOG and then
reset the CRC. (fsck) (8-byte overhead)

Both of which can be optimised in various ways.

Also, we might

* Put all hint bits in the block header to allow them to be excluded
more easily from CRC checking. If we used 3 more bits from
ItemIdData.lp_len (limiting tuple length to 4096) then we could store
some hints in the item pointer. HEAP_XMIN_INVALID can be stored as
LP_DEAD, since that will happen very quickly anyway. 

-- Simon Riggs           www.2ndQuadrant.com



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: CommitFest status/management
Next
From: Robert Haas
Date:
Subject: Re: Block-level CRC checks