Re: removing PD_ALL_VISIBLE - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: removing PD_ALL_VISIBLE
Date
Msg-id 20130531192256.GX6434@tamriel.snowman.net
Whole thread Raw
In response to Re: removing PD_ALL_VISIBLE  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers
* Bruce Momjian (bruce@momjian.us) wrote:
> Right, and it is hard to see that losing and adding are somehow
> more/less likely, so it seems we already realy on the visibility map
> being correct.

Yes, but there's also a way to get *back* to a valid state if things go
south with the visibility map.  It's similar to an index today- sure,
you might get corruption, and that might give you wrong results, but if
you detect it, it's easy to correct- rebuild the index, drop the
visibility map, etc.  With CRCs, it'll be even easier to detect and
discover when you need to do such a thing.

Corruption in the heap is certainly bad too- you may lose some records,
but you could zero those pages out and move on with only losing 8k or
what-have-you.  Clog corruption is certainly a bad thing, but if nearly
everything in your DB is already frozen, it's not as bad as it *could*
be.  I wonder if you could rebuild a portion of clog from the WAL..

There are a lot of different tradeoffs here, certainly.  It's certainly
nice that single bit or single page corruption can often be recovered
from by rebuilding an index or rebuilding the visbility map or just
nuking a few records from a table.  CRCs help with identifying when
corruption has happened, but they do nothing for what to *do* when it
happens, and "restore completely from backup" isn't a great answer when
you've got terrabytes of data.

Where I'm going with this whole thing is simply that I do worry a bit
about using a bitmap for freeze, or similar, information and not being
able to reconstruct that bitmap from the heap.  Perhaps that's overly
paranoid, but, well, we also write the same data out to disk in multiple
places multiple times- some might call that paranoid too. ;)
Thanks,        Stephen

pgsql-hackers by date:

Previous
From: Joe Conway
Date:
Subject: Re: detecting binary backup in progress
Next
From: Heikki Linnakangas
Date:
Subject: Re: detecting binary backup in progress