Re: pg_filedump 9.3: checksums (and a few other fixes) - Mailing list pgsql-hackers

From Andres Freund
Subject Re: pg_filedump 9.3: checksums (and a few other fixes)
Date
Msg-id 20130627135953.GJ1254@alap2.anarazel.de
Whole thread Raw
In response to Re: pg_filedump 9.3: checksums (and a few other fixes)  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: pg_filedump 9.3: checksums (and a few other fixes)  (Jeff Davis <pgsql@j-davis.com>)
List pgsql-hackers
On 2013-06-27 09:51:07 -0400, Tom Lane wrote:
> Andres Freund <andres@2ndquadrant.com> writes:
> > On 2013-06-26 21:18:49 -0700, Peter Geoghegan wrote:
> >> Heroku are interested in online verification of basebackups (i.e.
> >> using checksums to verify the integrity of heap files as they are
> >> backed up, with a view to relying less and less on logical backups).
> 
> > Why not do this from a function/background worker in the backend where
> > you can go via the buffer manager to avoid torn pages et al. If you use
> > a buffer strategy the cache poisoning et al should be controlleable.
> 
> That would require having a functioning postmaster environment, which
> seems like it would make such a tool much less flexible.

I personally wouldn't trust online backups that aren't proven to be
replayable into a runnable state very much. I have seen too many cases
where that failed.

Maybe the trick is to add a recovery.conf option to make postgres replay
to the first restartpoint and then shutdown. At that point you can be
sure there aren't any torn pages anymore (bugs aside).
In fact that sounds like a rather useful pg_basebackup extension...

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: David Fetter
Date:
Subject: Re: FILTER for aggregates [was Re: Department of Redundancy Department: makeNode(FuncCall) division]
Next
From: Robert Haas
Date:
Subject: Re: Add more regression tests for CREATE OPERATOR