Zeugswetter Andreas SB wrote:
>
> > > I don't like that --- seems it would put a definite crimp in the
> > > whole point of TOAST, which is not to have arbitrary limits on field
> > > sizes.
> >
> > If we can solve it, let's do so. If we cannot, let's restrict
> > it for 7.1.
>
> How are you doing the index toasting currently ? Is it on the same
> line as table toasting ? That is: toast some index column values if the key
> exceeds 2k ?
The current CVS is broken in that area. You'll notice as soon as you have many huge "text" values in an index,
updatethem, vacuum and continue to update.
The actual behaviour of the toaster is to toast each tuple until it has a delicious looking, brown and crispy
surface. The indicator for beeing delicious is that it shrank below MaxTupleSize/4 - that's a little less than 2K
ina default 8K blocksize setup.
It then sticks the new tuple into the HeapTuple's t_data pointer.
Index inserts are allways done after heap_insert() or heap_update(). At that time, the index tuples will
bebuilt from the values found in the now replaced heap tuple. And since the heap tuple found now is allways
smallerthan 2K, any combination of attributes out of it must be too (it's impossible to specify one and the
sameattribute multiple times in one index).
So the indices simply inherit the toasting result. If a value got compressed, the index will store the
compressedformat. If it got moved off, the index will hold the toast entry reference for it.
One of the biggest advantages is this: In the old system, an indexed column of 2K caused 2K be stored in the heap
plus 2K stored in the index. Plus all the 2K instances in upper index block range specs. Now, the heap and the
index will only hold references or compressed items.
Absolutely no problem for compressed items. All information to recreate the original value is in the Datum
itself.
For external stored ones, the reference tells the OIDs of the secondary relation and it's index (where to find
thedata of this entry), a unique identifier of the item (another OID) and some other info. So the
referencecontains all the information required to fetch the data just by looking at the reference. And since the
detoaster scans the secondary relation with a visibility of SnapShotAny, it'll succeed to find them even if
they'vebeen deleted long ago by another committed transaction. So index traversal will succeed on that in any
case.
What I didn't knew at the time of implementation is, that btree indices can keep such a reference in upper
levelblocks range specifications even after a vacuum successfully deleted the index tuple holding the reference
itself. That's the current pity.
Thus, if vacuum finally removed deleted tuples from the secondary relations (after the heap and index
have been vacuumed), the detoaster cannot find those entries, referenced by upper index blocks, any
more.
Maybe we could propagate key range changes into upper blocks at index_delete() time. Will look at the btree code
now.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
