Re: Create TOAST table only if AM needs - Mailing list pgsql-hackers

On Fri, May 17, 2019 at 2:34 PM Andres Freund <andres@anarazel.de> wrote:
> Yea, Robert is also working on this. In fact, we were literally chatting
> about it a few minutes ago. He'll probably chime in too.

Yeah, I'm aiming to post a patch set very soon that does a bunch of
refactoring of the TOAST stuff to make life easier for new AMs - maybe
today, or else Monday. I think your use case of wanting to suppress
TOAST table creation altogether is a valid one, but I want to do go a
little further and make it easier for non-heap AMs to implement
heap-like toasting based on their own page format.

Generally, I would say that the state of play with respect to table
AMs and toasting is annoyingly bad right now:

- If your AM uses some system other than TOAST to store large values,
you are out of luck.  You will get TOAST tables whether you want them
or not.

- If your AM uses some page or tuple format that results in a
different maximum tuple size, you are out of luck.  You will get TOAST
tables based on whether a heap table with the same set of columns
would need one.

- If your AM would like to use the heap for TOAST data, you are out of
luck.  The AM used for TOAST data must be the same as the AM used for
the main table.

- If your AM would like to use itself to store TOAST data, you are
also out of luck, because all of the existing TOAST code works with
heap tuples.

- Even if you copy all of tuptoaster.c/h - which is a lot of code -
and change everything that is different for your AM than for the
regular heap, you are still out of luck, because code that knows
nothing about tableam is going to call heap_tuple_untoast_attr() to
detoast stuff, and that code is only going to be happy if you've used
the same chunk size that we use for the regular heap, and that chunk
size has a good chance of being mildly to severely suboptimal if your
heap has made any sort of page format changes.

So I think this basically just doesn't work right now.  I am
sympathetic to Andres's position that we shouldn't go whacking the
code around too much at this late date, and he's probably right that
we're going to find lots of other problems with tableam as well and
you have to draw the line someplace, but on the other hand given your
experience and mine, it's probably pretty likely that anybody who
tries to use tableam for anything is going to run into this problem,
so maybe it's not crazy to think about a few last-minute changes.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company