[PATCH] Verify Checksums during Basebackups - Mailing list pgsql-hackers

From Michael Banck
Subject [PATCH] Verify Checksums during Basebackups
Date
Msg-id 20180228180856.GE13784@nighthawk.caipicrew.dd-dns.de
Whole thread Raw
Responses Re: [PATCH] Verify Checksums during Basebackups  (David Steele <david@pgmasters.net>)
Re: [PATCH] Verify Checksums during Basebackups  (Magnus Hagander <magnus@hagander.net>)
Re: [PATCH] Verify Checksums during Basebackups  (Michael Banck <michael.banck@credativ.de>)
List pgsql-hackers
Hi,

some installations have data which is only rarerly read, and if they are
so large that dumps are not routinely taken, data corruption would only
be detected with some large delay even with checksums enabled.

The attached small patch verifies checksums (in case they are enabled)
during a basebackup. The rationale is that we are reading every block in
this case anyway, so this is a good opportunity to check them as well.
Other and complementary ways of checking the checksums are possible of
course, like the offline checking tool that Magnus just submitted.

It probably makes sense to use the same approach for determining the
segment numbers as Magnus did in his patch, or refactor that out in a
utility function, but I'm sick right now so wanted to submit this for
v11 first.

I did some light benchmarking and it seems that the performance
degradation is minimal, but this could well be platform or
architecture-dependent. Right now, the checksums are always checked but
maybe this could be made optional, probably by extending the replication
protocol.


Michael

-- 
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax:  +49 2166 9901-100
Email: michael.banck@credativ.de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Attachment

pgsql-hackers by date:

Previous
From: Grigory Smolkin
Date:
Subject: Re: Changing the autovacuum launcher scheduling; oldest table firstalgorithm
Next
From: Robert Haas
Date:
Subject: Re: Server won't start with fallback setting by initdb.