CustomScan support on readfuncs.c - Mailing list pgsql-hackers

From Kouhei Kaigai
Subject CustomScan support on readfuncs.c
Date
Msg-id 9A28C8860F777E439AA12E8AEA7694F80114A8EE@BPXM15GP.gisp.nec.co.jp
Whole thread Raw
Responses Re: CustomScan support on readfuncs.c  (Amit Kapila <amit.kapila16@gmail.com>)
Re: CustomScan support on readfuncs.c  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
Hi,

I tried to define two additional callbacks to support CustomScan
on readfuncs.c.

First of all, we need to pay attention how to treat output of
TextOutCustomScan when additional text output is generated.
Only custom-scan provider knows what shall be displayed, so
all we can do is invoke a new callback: TextReadCustomScan().
Because private fields shall be displayed next to the common
field of CustomScan, this callback is invoked once pg_strtok
pointer reached to the last field of CustomScan. Then, custom
scan provider allocates CustomScan or a structure embedding
CustomScan; only CSP knows exact size to be allocated.
It can fetch some tokens using pg_strtok and reconstruct its
private fields, but no need to restore the common field because
it shall be done by _readCustomScan.
If no callbacks are defined, _readCustomScan allocates
a CustomScan object and read only known fields.

Then, let's look back a bit. Next issue is how to reproduce
the "methods" pointer from the text representation.
I try to lookup the methods table using a pair of library
and symbol name; probably, it is a straightforward way.
The "methods" field is put earlier than all private fields
generated by TextOutCustomScan, so it should be reconstructable
prior to TextReadCustomScan.
To support this feature, I had to add an interface contract
that requires extensions to put library and symbol name on
CustomScanMethods table.
Although INIT_CUSTOM_SCAN_METHODS() macro can set up these
fields, author of extension needs to pay attention.

In addition to these enhancement, a new NodeCopyCustomScan
callback eliminates a restriction; custom-scan provider
cannot define its own structure that embeds CustomScan.
Only CSP knows exact size of the structure, this callback
is intended to allocate a new one and copy the private fields,
but no need to copy the common fields.

These three callbacks (one existing, two new) will make
CustomScan node copyObject, nodeToString and stringToNode
aware.


I also noticed that some static functions are valuable for
extensions, to avoid reinvention of same routines.
How about to expose these functions?
- _outToken
- _outBitmapset
- nullable_string
- _readBitmapset


The attached patch is conceptual, just compilable but not
tested yet. I'll try to make a test case soon, however,
it makes sense to get feedbacks earlier even if it is
based on the concept design.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>

Attachment

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: DBT-3 with SF=20 got failed
Next
From: David Rowley
Date:
Subject: Re: PATCH: use foreign keys to improve join estimates v1