Re: Support logical replication of DDLs - Mailing list pgsql-hackers

On Tue, May 2, 2023 at 8:30 AM shveta malik <shveta.malik@gmail.com> wrote:
>
> On Fri, Apr 28, 2023 at 5:11 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > Now, I think we can try to eliminate this entire ObjTree machinery and
> > directly from the JSON blob during deparsing. We have previously also
> > discussed this in an email chain at [1]. I think now the functionality
> > of JSONB has also been improved and we should investigate whether it
> > is feasible to directly use JSONB APIs to form the required blob.
>
> +1.
> I will investigate this and will share my findings.
>


Please find the PoC patch for create-table after object-tree removal.
It is based on the patch dated May
2(ddl-replication-2023_05_02.tar.gz). The patches from 0001-0007 are
all the same, the patch '0008' is the new patch for objTree removal.
It is a WIP patch, sharing it for early feedback on the design part.
It covers most of the parts for create-table except a few things (eg:
with-clause, inheritance, owner) which are under implementation.

Regarding testing, some basic tests are done, extensive testing to be
done once we finish the patch. There are some expected diffs in
'test_ddl_deparse_regress' due to lack of complete create-table's
deparsing implementation yet.

The crux of this patch:
Earlier it was: ParseTree->Objtree->Jsonb->ddlMessage
Now it will be: ParseTree->Jsonb->ddlMessage

Some insights on how the object-tree removal is done:

In the current ObjTree implementation, we create a new tree of type
'ObjTree' and add 'ObjElem'(s) to it. ObjTree is basically a linked
list of ObjElem(s), plus some additional information which tells 'fmt'
and whether the concerned clause is 'present' in the given query or
not. Each ObjTree can further have another ObjTree as its child.
ObjElem OTOH has 3 main components:
1) name of element  (indicating clause's name)
2) value of element (indicating clause's value)
3) type of 'value'  (string,bool,array etc)

While conversion from ObjTree to jsonb, ObjTree maps to JsonbValue of
type 'jbvObject' and each of its ObjElem(s) maps to a pair of
'JsonbValue'(s) of required type.  Basically for each ObjElem created,
we create 2 'JsonbValue' structures. One structure is for the name of
'ObjElem' while another structure is for the 'value' of ObjElem,
'type' same as ObjElem's type. So the mapping goes as:

ObjElem  -->  JsonbValue of type jbvString for name of ObjElem +
JsonbValue of required type(same as type of 'value') for 'value' of
ObjElem. These together form a 'JsonbPair'.

ObjTree   --> JsonbValue of type jbvObject having all the JsonbPair(s)
contained in it.  'fmt:string' and 'present:true/false' are 2
key:Value pairs in each jbvObject.

Above means, if we want to remove ObjTree creation, wherever we were
creating ObjTree, we now need to create JsonbValue of type
'jbvObject' and wherever we were creating ObjElem, we now need to
create 2 JsonbValue structures for the 'name' and 'value' respectively
' and add those to the parent JsonbValue(type=jbvObject) structure.

We push all these jsonbValue to JsonbParseState using 'pushJsonbValue'
calls. Later these JsonbValue(s) are converted to Jsonb using '
JsonbValueToJsonb' which is then converted to string using
'JsonbToCString' and logged into WAL.
There is no change in this particular flow.

thanks
Shveta