Re: [RFC] indirect toast tuple support - Mailing list pgsql-hackers

On 2013-02-19 09:12:02 -0500, Robert Haas wrote:
> On Tue, Feb 19, 2013 at 9:00 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> >> So the other way that we could do this is to use something that's the
> >> same size as a TOAST pointer but has different content - the
> >> seemingly-obvious choice being  va_toastrelid == 0.
> >
> > Unfortunately that would mean you need to copy the varatt_external (or
> > whatever it would be called) to aligned storage to check what it
> > is. Thats why I went the other way.
> 
> How big a problem is that, though?

There are quite some places where we test the actual type of a Datum
inside tuptoaster.c. Copying it to local storage everytime might
actually be noticeable performancewise. Besides the ugliness of needing
a local variable, copying the data and only testing afterwards...

> >>  I'd be a little
> >> reluctant to do it the way you propose because we might, at some
> >> point, want to try to reduce the size of toast pointers.   If you have
> >> a tuple with many attributes, the size of the TOAST pointers
> >> themselves starts to add up.  It would be nice to be able to have 8
> >> byte or even 4 byte toast pointers to handle those situations.  If we
> >> steal one or both of those lengths to mean "the data is cached in
> >> memory somewhere" then we can't use those lengths in a smaller on-disk
> >> representation, which would seem a shame.
> >
> > I agree. As I said above, having the type overlayed into the lenght was
> > and is a bad idea, I just haven't found a better one thats compatible
> > yet.
> > Except inventing typlen=-3 aka "toast2" or something. But even that
> > wouldn't help getting rid of existing pg_upgraded tables. Besides being
> > a maintenance nightmare.
> >
> > The only reasonable thing I can see us doing is renaming
> > varattrib_1b_e.va_len_1be into va_type and redefine VARSIZE_1B_E into a
> > switch that maps types into lengths. But I think I would put this off,
> > except placing a comment somewhere, until its gets necessary.
> 
> I guess I wonder how hard we think it would be to insert such a thing
> when it becomes necessary.  How much stuff is there out there that
> cares about the fact that that length is a byte?

You mean whether we could store the length in 6 bytes and use two for
the type? That should probably work as well. But I don't see much
advantage in that given that all those sizes ought to be static.
Redefining VARSIZE_1B_E as indicated above should be fairly easy, there
aren't many callsites that touch stuff at such low level.

Greetings,

Andres Freund