Re: pg_dump include/exclude data, was: verify checksums / CREATEDATABASE - Mailing list pgsql-general

From Karsten Hilbert
Subject Re: pg_dump include/exclude data, was: verify checksums / CREATEDATABASE
Date
Msg-id 20190611071546.GB2660@hermes.hilbert.loc
Whole thread Raw
In response to Re: pg_dump include/exclude data, was: verify checksums / CREATEDATABASE  (Adrian Klaver <adrian.klaver@aklaver.com>)
Responses Re: pg_dump include/exclude data, was: verify checksums / CREATEDATABASE
List pgsql-general
Dear Adrian,

On Fri, Jun 07, 2019 at 08:02:32AM -0700, Adrian Klaver wrote:

> On 6/6/19 6:50 AM, Karsten Hilbert wrote:
> > The current canonical solution (?) for verifying checksums in
> > an existing database is, to may understanding, to pg_dump it
> > (to /dev/null, perhaps):
> >
> >     pg_dump --username=... --dbname=... --compress=0 --no-sync --format=custom --file=/dev/null
> >
> > as that will read and verify all blocks related to the dump
> > of that database.
>
> The above works with the existing cluster, but would you not also want to
> verify that the blocks written to on the new cluster also are good?

Sure, but that much depends on what (or rather, where) the
"new cluster" actually is.

In my case I want to make sure that - before a run of "CREATE
DATABASE new TEMPLATE old" - I can be reasonable sure that
the disk blocks underlying "old" verify against their
checksum. Since the CREATE DATABASE does not verify CRCs I
use a pg_dump into /dev/null to verify checksums while the
cluster is *online*.

The problem I hope to protect against with this approach: the
CREATE DATABASE might untaint corrupted data from a bad disk
block into a good disk block virtue of doing a file level
copy.

I hope my reasoning isn't going astray.

> > One will be tempted to include options to speed up the
> > process, say:
...
> > Is my assumption wrong ?
>
> Not sure, though it would seem to me including the above is a relatively
> small incremental cost to the overall dump, assuming a data set of any size
> greater then small.

Agreed. Good putting of things into perspective. Will do.

Thanks,
Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B



pgsql-general by date:

Previous
From: Rob Sargent
Date:
Subject: Re: Inserting into the blob
Next
From: Karsten Hilbert
Date:
Subject: Re: pg_dump include/exclude data, was: verify checksums / CREATEDATABASE