Re: pg_rawdump - Mailing list pgsql-hackers

From Tom Lane
Subject Re: pg_rawdump
Date
Msg-id 29907.1287594585@sss.pgh.pa.us
Whole thread Raw
In response to Re: pg_rawdump  (Aidan Van Dyk <aidan@highrise.ca>)
Responses Re: pg_rawdump
Re: pg_rawdump
List pgsql-hackers
Aidan Van Dyk <aidan@highrise.ca> writes:
> On Wed, Oct 20, 2010 at 10:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I can see the potential usefulness of a self-documenting table storage
>> format, but this proposal isn't that; it's just an unreliable kluge.

> If we're looking to have any sort of "out of catalog" documentation of
> table storage format, what about just having a new relation fork that
> just "appends" each and every change made to the table formats,
> including ones rolled back, etc.

I thought of that too, but I'm not sure if it's going to help enough.
The trouble is that the information is only tied to the table itself
via file names.  In a fsck-recovery scenario, you may not have the
correct file names.  (One of the multiple problems with Stephen's
proposal is that the metadata would only be physically tied to the
first segment of a large table, and unless you can trust the file
names you won't be able to match it to the other segments.)

[ thinks for a bit... ]  Perhaps we could stick some sort of unique
ID into tables, which could be correlated to the same unique ID
appearing in a metadata fork.

[ thinks some more... ]  Of course, at this point you have to start
asking why the metadata fork should be thought to be any more reliable
than the system catalogs.

> Make this relation fork append only,
> and dump a completely new set of metadata to it each and every ALTER
> TABLE.

You can bet that there'll be somebody out there who whines because their
app does lots and lots of repeated ALTER TABLEs, and so the metadata fork
grows forever.  I think we'd have to be a bit smarter than this.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Robert Treat
Date:
Subject: Re: Simplifying replication
Next
From: Florian Weimer
Date:
Subject: Re: patch: Add JSON datatype to PostgreSQL (GSoC, WIP)