Re: Prototype: In-place upgrade v02 - Mailing list pgsql-hackers

Zdenek Kotala wrote:
> Heikki Linnakangas napsal(a):
>> The patch seems to be missing the new htup.c file.
> 
> I'm sorry. I attached new version which is synchronized with current
> head. I would like to say more comments as well.
> 
> 1) The patch contains also changes which was discussed during July 
> commit fest.     - PageGetTempPage modification suggested by Tom
> - another hash.h backward compatible cleanup

It might be a good idea to split that into a separate patch. The sheer 
size of this patch is quite daunting, even though the bulk of it is 
straightforward search&replace.

> 2) I add tuplimits.h header file which contains tuple limits for 
> different access method. It is not finished yet, but idea is to keep all 
> limits in one file and easily add limits for different page layout 
> version - for example replace static computing with dynamic based on 
> relation (maxtuplesize could be store in pg_class for each relation).
> 
> I need this header also because I fallen in a cycle in header dependency.
> 
> 3) I already sent Page API performance result in 
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg00398.php
> 
> I replaced call sequence PagetGetItemId, PageGetItemId with 
> PageGetIndexTuple and PageGetHeapTuple function. It is main difference 
> in this patch. PAgeGetHeap Tuple fills t_ver in HeapTuple to identify 
> correct tupleheader version.
>
> It would be good to mention that PageAPI (and tuple API) implementation 
> is only prototype without any performance optimization.

You mentioned 5% performance degradation in that thread. What test case 
was that? What would be a worst-case scanario, and how bad is it?

5% is a pretty hefty price, especially when it's paid by not only 
upgraded installations, but also freshly initialized clusters. I think 
you'll need to pursue those performance optimizations.

> 4) This patch contains more topics for decision. First is general if 
> this approach is acceptable.

I don't like the invasiveness of this approach. It's pretty invasive 
already, and ISTM you'll need similar switch-case handling of all data 
types that have changed the internal representation as well.

We've talked about this before, so you'll remember that I favor teh 
approach is to convert the page format, page at a time, when the pages 
are read in. I grant you that there's non-trivial issues with that as 
well, like if the converted data takes more space and don't fit in the 
page anymore.

I wonder if we could go with some sort of a hybrid approach? Convert the  whole page when it's read in, but if it
doesn'tfit, fall back to 
 
tricks like loosening the alignment requirements on platforms that can 
handle non-aligned data, or support a special truncated page header, 
without pd_tli and pd_prune_xid fields. Just a thought, not sure how 
feasible those particular tricks are, but something along those lines..

All in all, though. I find it a bit hard to see the big picture. For 
upgrade-in-place, what are all the pieces that we need? To keep this 
concrete, let's focus on PG 8.2 -> PG 8.3 (or are you focusing on PG 8.3 
-> 8.4? That's fine with me as well, but let's pick one) and forget 
about hypothetical changes that might occur in a future version. I can see:
1. Handling page layout changes (pd_prune_xid, pd_flags)
2. Handling tuple header changes (infomask2, HOT bits, combocid)
3. Handling changes in data type representation (packed varlens)
4. Toast chunk size
5. Catalogs

After putting all those together, how large a patch are we talking 
about, and what's the performance penalty then? How much of all that 
needs to be in core, and how much can live in a pgfoundry project or an 
extra binary in src/bin or contrib? I realize that none of us have a 
crystal ball, and one has to start somewhere, but I feel uneasy 
committing to an approach until we have a full plan.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com