Postgres Pro
Downloads
Home   >   mailing lists

Re: Pluggable toaster - Mailing list pgsql-hackers

From Nikita Malakhov
Subject Re: Pluggable toaster
Date
Msg-id CAN-LCVNj0iLHRA4CzZHk3-BEt2-47Xbx1h020ReB0LA-=5H9YA@mail.gmail.com
Whole thread Raw
In response to Re: Pluggable toaster  (Teodor Sigaev <teodor@sigaev.ru>)
Responses Re: Pluggable toaster  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Tree view
Hi Hackers,
Because of 3 months have passed since Pluggable Toaster presentation and a lot of 
commits were pushed into v15 master - we would like to re-introduce this patch 
rebased onto actual master. Last commit being used -
commit 641f3dffcdf1c7378cfb94c98b6642793181d6db (origin/master)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Fri Mar 11 13:47:26 2022 -0500

Updated patch consists of 4 patch files, next version (v2) of original patch files
(please check original commit message from 30 Dec 2020):
1) 1_toaster_interface_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_interface
Introduces  syntax for storage and formal toaster API.

2) 2_toaster_default_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_default
Built-in toaster implemented (with some refactoring)  uisng toaster API
as generic (or default) toaster.

3) 3_toaster_snapshot_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_snapshot
The patch implements technology to distinguish row's versions in toasted
values to share common parts of toasted values between different
versions of rows

4) 4_bytea_appendable_toaster_v2.patch.gz
https://github.com/postgrespro/postgres/tree/bytea_appendable_toaster
Contrib module implements toaster for non-compressed bytea columns,
which allows fast appending to existing bytea value.

These patches also include 2 minor fixes made after commit fest presentation
1) Fix for freeing memory in case of new toasted value is the same as old one, 
this seems incorrect, and in this case the function just returns instead of freeing old value;

2) Fix of data alignment for new varatt_custom data structure in building tuples, 
since varatt_custom must be aligned for custom toasters (in particular, this fix is very 
important to JSONb Toaster).

Thanks!

On Wed, Feb 2, 2022 at 10:35 AM Teodor Sigaev <teodor@sigaev.ru> wrote:
> I agree ... but I'm also worried about what happens when we have
> multiple table AMs. One can imagine a new table AM that is
> specifically optimized for TOAST which can be used with an existing
> heap table. One can imagine a new table AM for the main table that
> wants to use something different for TOAST. So, I don't think it's
> right to imagine that the choice of TOASTer depends solely on the
> column data type. I'm not really sure how this should work exactly ...
> but it needs careful thought.

Right. that's why we propose a validate method  (may be, it's a wrong
name, but I don't known better one) which accepts several arguments, one
of which is table AM oid. If that method returns false then toaster
isn't useful with current TAM, storage or/and compression kinds, etc.

--
Teodor Sigaev                      E-mail: teodor@sigaev.ru
                                       WWW: http://www.sigaev.ru/




--
Regards,
Nikita Malakhov
Postgres Professional 
Attachment

pgsql-hackers by date:

Previous
From: Mark Dilger
Date:
Subject: Re: New Object Access Type hooks
Next
From: Andres Freund
Date:
Subject: Re: PoC: using sampling to estimate joins / complex conditions