Re: Pluggable toaster - Mailing list pgsql-hackers
| From | Nikita Malakhov |
|---|---|
| Subject | Re: Pluggable toaster |
| Date | |
| Msg-id | CAN-LCVM3zP2SrdFksF3p7sk_hFX_NE081F=r1iT_H39QQp1YkA@mail.gmail.com Whole thread Raw |
| In response to | Re: Pluggable toaster (Aleksander Alekseev <aleksander@timescale.com>) |
| Responses |
Re: Pluggable toaster
(Aleksander Alekseev <aleksander@timescale.com>)
|
| List | pgsql-hackers |
Hi,
>I really think that pluggable TOASTers would be a great feature.
>However if the goal is to get it into the core I doubt that we are
>going to make much progress with the current approach.
>This is exactly the point. In order to not to create a silver bullet,
>TOASTers should be limited to a single TableAM. The one we know uses
>pages of a known fixed size, the one that actually requires TOAST
>because pages are relatively small, etc.
>TOASTers should be limited to a single TableAM. The one we know uses
>pages of a known fixed size, the one that actually requires TOAST
>because pages are relatively small, etc.
Currently all our TOAST implementations use Heap AM, except ones
that use external (truly external, i.e. files outside DB) storage. Using Table AM
Routine and routing AM methods calls via it is a topic for further discussion,
if Pluggable TOAST will be committed. And even then it would be an open issue.
>I believe this is the source of misunderstanding. Note that not _any_
>TableAM needs TOAST to begin with. As an example, if one chooses to
>implement a column-organized TableAM that stores all text/bytea
>attributes in a separate dictionary file this person doesn't need
>TOAST and doesn't want to be constrained by the need of choosing one.
>For this reason the "N TOASTers x M TableAMs" approach is
>architecturally broken.
>TableAM needs TOAST to begin with. As an example, if one chooses to
>implement a column-organized TableAM that stores all text/bytea
>attributes in a separate dictionary file this person doesn't need
>TOAST and doesn't want to be constrained by the need of choosing one.
>For this reason the "N TOASTers x M TableAMs" approach is
>architecturally broken.
TOAST implementation is not necessary for Table AM. And TOAST API is just
an optional open interface - SET TOASTER is an option for CREATE/ALTER
TABLE command. In previous discussion we haven't mentioned an approach
"N TOASTers x M TableAMs".
>To clarify: is an ability to specify TOASTers for given columns and/or
>types also part of the plan?
>types also part of the plan?
For table columns it is already supported by the syntax part of the TOAST API.
For data types we reserved the validation part of the API, but this support is still a
subject for discussion, although we think it will be very handy for DB users, like
we issue something like:
CREATE TYPE ... TOASTER=jsonb_toaster ... ;
or
ALTER TYPE JSONB SET TOASTER jsonb_toaster;
and do not have to set special toaster for jsonb column each time we create
or alter a table with it.
>No. To be honest, it looks like you are merely discarding most/any
>feedback the community provided so far.
Very sorry to read that. Almost all of the requests in this discussion have been taken
>feedback the community provided so far.
Very sorry to read that. Almost all of the requests in this discussion have been taken
into account in patches, and the most serious one - I mean pg_attribute expansion
which was mentioned by Tom Lane and Robert Haas - is being fixed right now and
will be ready very soon.
>I really think that pluggable TOASTers would be a great feature.
>However if the goal is to get it into the core I doubt that we are
>going to make much progress with the current approach.
We hope we will. This feature is very demanded by end-users, and will be even more
as time goes by - current TOAST limitations and how they affect DBMS performance is
a serious drawback in comparison to competitors.
On Mon, Oct 24, 2022 at 2:55 PM Aleksander Alekseev <aleksander@timescale.com> wrote:
Hi Nikita,
> >I don't argue with most of what you say. I am just pointing out the
> >reason why the chosen approach "N TOASTers x M TableAMs" will not
> >work:
>
> We assume that TAM used in custom Toaster works as it is should work,
> and leave TAM internals to this TAM developer - say, we do not want to
> change internals of Heap AM.
>
> We don't want to create some kind of silver bullet.
This is exactly the point. In order to not to create a silver bullet,
TOASTers should be limited to a single TableAM. The one we know uses
pages of a known fixed size, the one that actually requires TOAST
because pages are relatively small, etc.
> That's what the TOAST API is - just an interface that all custom
> TOAST implementations use to have a common entry point from any TAM,
> [...]
I believe this is the source of misunderstanding. Note that not _any_
TableAM needs TOAST to begin with. As an example, if one chooses to
implement a column-organized TableAM that stores all text/bytea
attributes in a separate dictionary file this person doesn't need
TOAST and doesn't want to be constrained by the need of choosing one.
For this reason the "N TOASTers x M TableAMs" approach is
architecturally broken.
> keep in mind that different kinds of data require very
> different approach to external storage - say, JSON TOAST works with
> maps of keys and values, [...]
To clarify: is an ability to specify TOASTers for given columns and/or
types also part of the plan?
> Have I answered your question? Please don't hesitate to point to any unclear
> parts, I'd be glad to explain that.
No. To be honest, it looks like you are merely discarding most/any
feedback the community provided so far.
I really think that pluggable TOASTers would be a great feature.
However if the goal is to get it into the core I doubt that we are
going to make much progress with the current approach.
--
Best regards,
Aleksander Alekseev
pgsql-hackers by date: