Re: Pluggable toaster - Mailing list pgsql-hackers
| From | Nikita Malakhov |
|---|---|
| Subject | Re: Pluggable toaster |
| Date | |
| Msg-id | CAN-LCVMxZU1F2E5NenDQ6JJDpdUkEK00feM7ih8ts6iqP3FF1w@mail.gmail.com Whole thread Raw |
| In response to | Re: Pluggable toaster (Jacob Champion <jchampion@timescale.com>) |
| Responses |
Re: Pluggable toaster
|
| List | pgsql-hackers |
Hi hackers!
Jacob, I agree that the bytea toaster makes a bad example due to core modification,
and actually is not a good example of an extension.
The vtable concept is intended for less invasive additional functionality - like providing
detoast iterators in addition to standard detoast mechanics - such modification requires
only adding iteration methods to toaster and registering them in vtable, without any
core modifications. I'll add this as a separate commit for generic (default) Toaster.
It would be more clear for complex data types like JSONB, where developers could
need some additional functionality to work with internal representation of data type,
and its full potential is revealed in our JSONB toaster extension. The JSONB toaster
is still in development but we plan to make it available soon.
For example, we can pass Toaster options with attoptions (I'm currently working on it)
and these options could, say, allow switching different optimizations in one toaster like
adding specific compression options or data processing directives, etc.
We doubt that there would be a lot of different custom toasters, because the Toaster
is quite a complex piece of machinery, but means for extending them would be heavily
demanded. I have to add some more in-depth explanation of the vtable concept to
README and the documentation package, the dummy toaster contrib does not cover
this topic at all.
On installing dummy_toaster contrib: I've just checked it by making a patch from commit
and applying onto my clone of master and 2 patches provided in previous email without
any errors and sll checks passed - applying with git am, configure with debug, cassert,
depend and enable-tap-tests flags and run checks.
Please advice what would cause such a behavior?
Thank you!
On Tue, Sep 13, 2022 at 12:39 AM Jacob Champion <jchampion@timescale.com> wrote:
On Wed, Aug 24, 2022 at 2:59 AM Nikita Malakhov <hukutoc@gmail.com> wrote:
> I've rebased actual branch onto the latest master and re-created patches. Checked with git am,
> all applied correctly. Please check the attached patches.
> Rebased branch resides here:
> https://github.com/postgrespro/postgres/tree/toasterapi_clean
I tried installing and using the dummy_toaster that's provided with
the gitlink. Upgrade of that cluster fails with the following message:
pg_restore: creating TOASTER "dummy_toaster"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 2044; 9861 16390 TOASTER dummy_toaster (no owner)
pg_restore: error: could not execute query: ERROR: unrecognized
or unsupported class OID: 9861
Command was: CREATE TOASTER "dummy_toaster" HANDLER
"public"."dummy_toaster_handler";
I was looking through the thread for a more in-depth description of
the "vtable" concept, but I didn't see one. It looks like it's just an
arbitrary extension point, and any new additions would require surgery
on whatever function needs the particular magic provided by the
toaster. E.g. your bytea-append toaster extension in the gitlink,
which still has to modify byteacat() in varlena.c to implement a very
specific optimization, and then declares its support for that
hardcoded optimization in the extension.
I'm skeptical that this would remain coherent as it grows. The patch
claims the vtable API is "powerful", which... I suppose it is, if you
get to make arbitrary modifications to the core whenever you implement
it. Did you already have thoughts about which operations would belong
under that umbrella? What would the procedure be for adding
functionality to that API? What happens if a toaster wants to
implement two magic performance optimizations instead of one?
--Jacob
pgsql-hackers by date: