Re: [GENERAL] Checksum and Wal files - Mailing list pgsql-general

From Stephen Frost
Subject Re: [GENERAL] Checksum and Wal files
Date
Msg-id 20170323142316.GT9812@tamriel.snowman.net
Whole thread Raw
In response to [GENERAL] Checksum and Wal files  (Poul Kristensen <bcc5226@gmail.com>)
List pgsql-general
Greetings,

* Poul Kristensen (bcc5226@gmail.com) wrote:
> Are all files inclusive wal files added a checksum?

WAL files include checksums, yes.  The heap files in recent versions can
also include checksums, but they don't by defualt.  Currently, checksums
can only be enabled at initdb-time, meaning that if you have a database
which doesn't have checksums today then you have to create a *new*
database, export all of the data from existing database using 'pg_dump'
and then import it into the database which has heap checksums enbaled.

> We will be doing regularly tests of recovery using checksums as the
> parameter
> of a succesfull recovery.

I'm curious what you mean by this statement.  There aren't many tools
out there today for checking if the heap checksums in PostgreSQL are
valid or not.  One approach is to use PostgreSQL itself and convince it
to read every page, though that can be a bit troublesome to do.  Another
option is to use a backup tool which checks checksums as it's doing the
backup (pgbackrest being one, and I think there may have been a fork of
barman which also did..?).

That said, I would strongly encourage you to find a better approach to
testing your backup and recovery procedures as just checking checksums
doesn't tell you if there's been any database-level corruption, it'll
only identify filesystem-and-below corruption (well, and it might catch
some database-level bugs, but the coverage there is exceedingly small).

Thanks!

Stephen

Attachment

pgsql-general by date:

Previous
From: Poul Kristensen
Date:
Subject: [GENERAL] Checksum and Wal files
Next
From: Subhankar Chattopadhyay
Date:
Subject: [GENERAL] Lag in asynchronous replication