I can't help with most of the question, but as I've implemented new
TOAST access methods, I can answer this part:
On Mon, 2002-01-28 at 21:51, Peter Eisentraut wrote:
>
> The question I'm particularly struggling with is, when does TOASTing and
> de-TOASTing happen? And if it doesn't, what's the official way to do it?
> I've found PG_DETOAST_DATUM and PG_DETOAST_DATUM_COPY. Why would I want a
> copy? (How can detoasting happen without copying?) And if I want a copy,
> in what memory context does it live? And can I just pfree() the copy if I
> don't want it any longer?
I think there are two contexts for detoasting.
1) fmgr functions. The PG_GETARG macro fetches the argument Datum and
passes it through PG_DETOAST_DATUM (if the Datum is a TOASTable type).
Thus the Datum from PG_GETARG_ is always detoasted.
2) Other access. I believe that heap_getattr will return a Datum -which
for TOASTable types will be a varlena struct. This may contain either
the literal data for the value (compressed or not) or the TOAST-pointer
(toastrelid, toastvalueid). These various cases are distinguished by the
top two bits of the varlena length field.
In all cases other than the "uncompressed, inline" case, the value must
be passed through PG_DETOAST_DATUM to guarantee a "standard" varlena
i.e. a value that is detoasted and stored in memory, can be accessed
directly from C etc.
However, the pointer returned by PG_DETOAST_DATUM might be *either* a
pointer to the original varlena struct, or to a decompressed value in
newly palloc'ed space. Thus the need for PG_DETOAST_DATUM_COPY, which
makes a copy of an uncompressed varlena, so that you can treat all the
cases in the same way. I believe that the detoasted datums from the
_COPY macros are ordinary things allocated by palloc in the current
memory context, so you can write to them and pfree() them if you wish.
The non-COPY variety might return a pointer to the inside of the tuple
data, which is not to be modified!
fmgr.h defines all the access methods, and also defines PG_FREE_IF_COPY,
which compares the pointer of the detoasted Datum to the original Datum
pointer and only calls pfree if they differ.
Hope this helps.
Regards
John
