Re: pg_upgrade: convert on read is dead end - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: pg_upgrade: convert on read is dead end
Date
Msg-id 48F2F310.9090303@enterprisedb.com
Whole thread Raw
In response to pg_upgrade: convert on read is dead end  (Zdenek Kotala <Zdenek.Kotala@Sun.COM>)
Responses Re: pg_upgrade: convert on read is dead end
List pgsql-hackers
Zdenek Kotala wrote:
> PostgreSQL stores all table's external data in one TOAST table and 
> stored data does not contains any clue about datatype. When postgreSQL 
> reads TOSTEed value then there is not possible detect what type it is 
> and perform varlena conversion on composite datatypes or arrays.
> 
> It could be converted in detoast_datum function but it means that datum 
> have to be retoasted and store back on new pages. The old page MUST keep 
> in old format because any page conversion lost information about version 
> and different datatypes can be store on one page PosgreSQL.

Hmm. There's two things here:
1. The format of the toast page, and the toast chunk tuples.
2. The format of the toasted tuple, stored within the chunk tuples.

The format of the toast page and the varlena headers of the byteas in 
the toast tuples could be converted when the page is read in, as usual. 
The format of the toasted tuple within the chunks is indeed trickier. 
Retoasting all tuples on the "master" heap page when the page is read in 
is probably simplest, as you say. But they don't necessarily need to be 
stored on new pages, any toast pages will do, because we keep track of 
1. separately.

> By my opinion, this issue completely kill convert on read solution and 
> only "read all formats..." solution is right one.

It is quite heavy-weight, I agree, but doesn't "completely kill" the 
idea in my opinion.

Besides, the "read all formats" approach wouldn't really avoid it 
either. If you passed a toasted datum to a function, when the function 
needs to detoast it, detoast_datum still wouldn't know whether the datum 
is in old or new format. You'd still need to detoast all values in the 
tuple somewhere before they could be passed around.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: "Hitoshi Harada"
Date:
Subject: Re: SELECT TOP %d PERCENT, or SELECT ... LIMIT %d PERCENT ?
Next
From: Zdenek Kotala
Date:
Subject: Re: [PATCH] Extending pg_class info + more flexible TOAST chunk size