Re: [v9.5] Custom Plan API - Mailing list pgsql-hackers

> Kouhei Kaigai <kaigai@ak.jp.nec.com> writes:
> > Let assume a custom-scan provider that runs on a materialized-view
> > (or, something like a query cache in memory) instead of join.
> > In this case, a reasonable design is to fetch a tuple from the
> > materialized-view then put it on the ecxt_scantuple of ExprContext
> > prior to evaluation of qualifier or tlist, unlike usual join takes two
> > slots - ecxt_innertuple and ecxt_outertuple.
> > Also, it leads individual varnode has to reference exct_scantuple,
> > neither ecxt_innertuple nor ecxt_outertuple.
>
> OK, that's possibly a reasonable way to do it at runtime.  You don't
> *have* to do it that way of course.  It would be only marginally less
> efficient to reconstruct two tuples that match the shapes of the original
> join inputs.
>
> > I intended to use the SetCustomScanRef callback to adjust varno and
> > varattno of the varnode that references the custom-scan node; as if
> > set_join_references() doing.
>
> I think this is really fundamentally misguided.  setrefs.c has no business
> doing anything "interesting" like making semantically important
> substitutions; those decisions need to be made much earlier.  An example
> in the context of your previous proposal is that getting rid of expensive
> functions without any adjustment of cost estimates is just wrong; and I
> don't mean that you forgot to have your setrefs.c hook hack up the Plan's
> cost fields.  The cost estimates need to change at the Path stage, or the
> planner might not even select the right path at all.
>
Because we right now have no functionality to register custom-scan path
instead of join, I had to show another use scenario...

> I'm not sure where would be an appropriate place to deal with the kind of
> thing you're thinking about here.  But I'm really not happy with the concept
> of exposing the guts of setrefs.c in order to enable single-purpose kluges
> like this.  We have fairly general problems to solve in this area, and we
> should be working on solving them, not on freezing relevant planner APIs
> to support marginally-useful external plugins.
>
From my standpoint, varnode remapping on relations join is higher priority
than complicated expression node. As long as the core backend handles this
job, yes, I think a hook in setrefs.c is not mandatory.
Also, it means the role to solve special vernode on EXPLAIN is moved from
extension to the code, GetSpecialCustomVar can be rid.

Let me explain the current idea of mine.
CustomScan node will have a field that hold varnode mapping information
that is constructed by custom-scan provider on create_customscan_plan,
if they want. It is probably a list of varnode.
If exists, setrefs.c changes its behavior; that updates varno/varattno of
varnode according to this mapping, as if set_join_references() does
based on indexed_tlist.
To reference exct_scantuple, INDEX_VAR will be a best choice for varno
of these varnodes, and index of the above varnode mapping list will
be varattno. It can be utilized to make EXPLAIN output, instead of
GetSpecialCustomVar hook.

So, steps to go may be:
(1) Add custom_private, custom_exprs, ... instead of self defined data   type based on CustomXXX.
(2) Rid of SetCustomScanRef and GetSpecialCustomVar hook for the current   custom-"scan" support.
(3) Integration of above varnode mapping feature within upcoming join   replacement by custom-scan support.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>