Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API) - Mailing list pgsql-hackers

> Where are we on this? AFAIK, we have now a feature with no documentation
> and no example in-core to test those custom routine APIs, hence moved to
> next CF.
>
Now Hanada-san is working on the example module that use this new
infrastructure on top of postgres_fdw. Probably, he will submit the
patch within a couple of days, for the upcoming commit fest.

Regarding to the documentation, a consensus was to make up a wikipage
to edit the description by everyone, then it shall become source of
SGML file.
The latest one is here:
https://wiki.postgresql.org/wiki/CustomScanInterface

Anyway, the next commit-fest shall within a couple of days. I'd like to
have discussion for the feature.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>


> -----Original Message-----
> From: Michael Paquier [mailto:michael.paquier@gmail.com]
> Sent: Friday, February 13, 2015 4:38 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Robert Haas; Tom Lane; pgsql-hackers@postgreSQL.org; Shigeru Hanada
> Subject: ##freemail## Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5]
> Custom Plan API)
>
>
>
> On Thu, Jan 15, 2015 at 8:02 AM, Kouhei Kaigai <kaigai@ak.jp.nec.com> wrote:
>
>
>     > On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai
> <kaigai@ak.jp.nec.com> wrote:
>     > > When custom-scan node replaced a join-plan, it shall have at
> least two
>     > > child plan-nodes. The callback handler of PlanCustomPath needs
> to be
>     > > able to call create_plan_recurse() to transform the underlying
>     > > path-nodes to plan-nodes, because this custom-scan node may
> take other
>     > > built-in scan or sub-join nodes as its inner/outer input.
>     > > In case of FDW, it shall kick any underlying scan relations
> to remote
>     > > side, thus we may not expect ForeignScan has underlying plans...
>     >
>     > Do you have an example of this?
>     >
>     Yes, even though full code set is too large for patch submission...
>
>     https://github.com/pg-strom/devel/blob/master/src/gpuhashjoin.
> c#L1880
>
>     This create_gpuhashjoin_plan() is PlanCustomPath callback of
> GpuHashJoin.
>     It takes GpuHashJoinPath inherited from CustomPath that has
> multiple
>     underlying scan/join paths.
>     Once it is called back from the backend, it also calls
> create_plan_recurse()
>     to make inner/outer plan nodes according to the paths.
>
>     In the result, we can see the following query execution plan that
> CustomScan
>     takes underlying scan plans.
>
>     postgres=# EXPLAIN SELECT * FROM t0 NATURAL JOIN t1 NATURAL JOIN
> t2;
>                                         QUERY PLAN
>     --------------------------------------------------------------
> --------------------
>      Custom Scan (GpuHashJoin)  (cost=2968.00..140120.31
> rows=3970922 width=143)
>        Hash clause 1: (aid = aid)
>        Hash clause 2: (bid = bid)
>        Bulkload: On
>        ->  Custom Scan (GpuScan) on t0  (cost=500.00..57643.00
> rows=4000009 width=77)
>        ->  Custom Scan (MultiHash)  (cost=734.00..734.00 rows=40000
> width=37)
>              hash keys: aid
>              nBatches: 1  Buckets: 46000  Memory Usage: 99.99%
>              ->  Seq Scan on t1  (cost=0.00..734.00 rows=40000
> width=37)
>              ->  Custom Scan (MultiHash)  (cost=734.00..734.00
> rows=40000 width=37)
>                    hash keys: bid
>                    nBatches: 1  Buckets: 46000  Memory Usage: 49.99%
>                    ->  Seq Scan on t2  (cost=0.00..734.00 rows=40000
> width=37)
>     (13 rows)
>
>
>
> Where are we on this? AFAIK, we have now a feature with no documentation
> and no example in-core to test those custom routine APIs, hence moved to
> next CF.
> --
>
> Michael