Re: Does PostgreSQL check database integrity at startup? - Mailing list pgsql-general

From Alban Hertroys
Subject Re: Does PostgreSQL check database integrity at startup?
Date
Msg-id 1B534695-9219-4555-B31E-92A73D390621@gmail.com
Whole thread Raw
In response to Re: Does PostgreSQL check database integrity at startup?  (Jan Wieck <jan@wi3ck.info>)
Responses Re: Does PostgreSQL check database integrity at startup?
List pgsql-general

> On 30 Dec 2017, at 6:09, Jan Wieck <jan@wi3ck.info> wrote:
>
> This whole discussion left out the critical point of
>
> how on earth did that file end up having zero bytes?
>
> There is the possibility that this is due to a bug in PostgreSQL. But over the 20+ years I've been using and hacking
it,that has never happened. 
>
> OP never told us what type of OS, hardware and specifically storage subsystem that "slave" was running on.
Filesystemszapping files to zero after a catastrophic IO failure (no longer throwing them into lost+found) isn't
unheardof. 

If this is indeed a case of a corrupt database, then it's possible that we're looking at a system that claims to have
thedisk cache flushed when it hasn't yet done that. 

It gets even more fun if there are multiple table-spaces in use with different behaviour in "whatever is causing this"
(fsync,file system zappery, etc.); the WAL log can get out of synch with the table files in that case. The WAL would
reflecta situation where more has been flushed to the respective files than is actually the case - data got lost. 

But of course it's also entirely possible that these empty files are not in fact a problem, as suggested up-thread.

No reason to panic - yet.
In fact, I don't see it mentioned explicitly anywhere, but are we actually looking at a problem?
The standby server started up just fine, from what I'm reading. Is there any evidence of corruption?

So far, the database has been labeled corrupt by Edson because a base file was 0 bytes, but that's apparently not a
definitiveindication of corruption. 
What would be definitive is if querying certain relations causes errors or does give results that were never in that
database.I do think that those 0 byte files are a good starting point, by querying the relations that those belong to
(ifany). Otherwise we may be searching for a needle in a haystack that doesn't necessarily contain one. 
I do realise that not finding any evidence of corruption does not necessarily mean there is none.

I'm not convinced by the argument that a CRC check of a 0 byte file on a standby would not detect corruption. At the
least,the CRC would be different or we would be left with a CRC on the master that we can't match any CRC's on the
slaveto if the file is larger on the master. 

If CRC's can be relied on to detect corruption (which they were designed to do), then that answers Edson's question.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.



pgsql-general by date:

Previous
From: Jan Wieck
Date:
Subject: Re: Does PostgreSQL check database integrity at startup?
Next
From: Stephen Frost
Date:
Subject: Re: Does PostgreSQL check database integrity at startup?