Re: Logical insert/update/delete WAL records for custom table AMs - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: Logical insert/update/delete WAL records for custom table AMs
Date
Msg-id 2c31c63033126a2882799fd619b52b147bb365d8.camel@j-davis.com
Whole thread Raw
In response to Re: Logical insert/update/delete WAL records for custom table AMs  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: Logical insert/update/delete WAL records for custom table AMs
List pgsql-hackers
On Thu, 2021-11-04 at 14:33 +0530, Amit Kapila wrote:
> Can't different tableAM's have different representations
> for toast or may be some different concept like speculative
> insertions?

The decoding plugin should just use the common interfaces to toast, and
if the table AM supports toast, everything should work fine. The only
special thing it needs to do is check VARATT_IS_EXTERNAL_ONDISK(),
because on-disk toast data can't be decoded (which is true for heap,
too).

I haven't looked as much into speculative insertions, but I don't think
those are a problem either. Shouldn't they be handled before they make
it into the change stream that the plugin sees?

> Similarly, I remember that for zheap we didn't had
> TransactionIds for subtransactions so we need to make some changes in
> logical decoding to compensate for that. I guess similar stuff could
> be required for another table AMs. Then a different table AM can have
> a different tuple format which won't work for current change records
> unless we convert it to heap tuple format before writing WAL for it.

Columnar certainly has a different format. That makes me wonder whether
ReorderBufferChange and/or ReorderBufferTupleBuf are too low-level, and
we should have a higher-level representation of a change that is based
on slots.

Can you tell me more about the changes you made for zheap? I still
don't understand why the decoding plugin would have to know what table
AM the change came from.

Regards,
    Jeff Davis





pgsql-hackers by date:

Previous
From: Mark Dilger
Date:
Subject: Re: Extending amcheck to check toast size and compression
Next
From: Jeff Davis
Date:
Subject: Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.