Re: WAL format - Mailing list pgsql-hackers

From Tom Lane
Subject Re: WAL format
Date
Msg-id 24224.1260218677@sss.pgh.pa.us
Whole thread Raw
In response to Re: WAL format  ("Kevin Grittner" <Kevin.Grittner@wicourts.gov>)
Responses Re: WAL format  (Andres Freund <andres@anarazel.de>)
Re: WAL format  (Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>)
List pgsql-hackers
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote:
>> In particular I wonder why we bother with the page headers.
> Since we re-use the file for a new segment, without overwriting the
> old contents, it seems like we would need to do *something* to
> reliably determine when we've hit the end of a segment and have
> moved into old data from a previous use of the file.  Would your
> proposed changes cover that adequately?

AFAICT the proposal would make us 100% dependent on the record CRC
to detect when a record has been torn (ie, only the first few sectors
made it to disk).  I'm a bit nervous about that from a reliability
standpoint --- with a 32-bit CRC you've got a 1-in-4-billion chance
of accepting bad data.  Checking the page headers too gives us many
more bits that have to be as-expected to consider the data good.

Since the records are fed to XLogInsert as units, it seems like the
actual problem might be addressable by hooking in the sync-rep data
sending at that level, rather than looking at the WAL page buffers
as I gather it must be doing now.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Build sizes vs docs
Next
From: Andres Freund
Date:
Subject: Re: WAL format