Re: Postgresql - FDW, ForeignScanState and subqueries - Mailing list pgsql-general

From Ronan Dunklau
Subject Re: Postgresql - FDW, ForeignScanState and subqueries
Date
Msg-id CAJWq4=Z4hQHZmB3yuEoyNc44w5NSLa_Uf0=nzqe-5NtQp-fuuA@mail.gmail.com
Whole thread Raw
In response to Re: Postgresql - FDW, ForeignScanState and subqueries  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
So, according to your advice, I've looked at the IndexScan implementation and here is what I came up with.
At execution time, if I find an operation expression involving a Param, the param expression is evaluated with ExecEvalExpr (after being "initted" on the fly), and then considered exactly the same as a Const->constvalue.

Is there any problem with that I should know about ?

I should maybe cache the resulting ExprState information during plan time, and only evaluate it during execution time. Would that make a huge difference ? 

Regarding the columns extraction information,  I'll try to move my code in the plan-time hook.

Thank you very much !

Best Regards,

--
Ronan Dunklau


2011/10/20 Tom Lane <tgl@sss.pgh.pa.us>
Ronan Dunklau <rdunklau@gmail.com> writes:
> Now, if I query my table like this (a subquery, joined on the outer query),
> what info should I be able to parse from the PlanState ?

> select name, (select max(value) from test t2 where t2.name = t1.name) as max
> from test t1;

> I don't really know much about postgresql internals, regarding execution
> plans, but here is what I guessed from what I managed to extract from the
> plan tree so far:

> - The outer query is executed once, restricting only the needed columns
> - The subquery is executed once for each row, with:
>    - all columns from the table are requested in a "target entry" node, even
> if the query only need the name and value columns.
>    - the value corresponding to the name from the outer query is somehow
> passed in as an Expr of type T_Param, in the quals field.

> How can I retrieve the value from the Param struct ?

You shouldn't be trying to do it at that level --- at most you ought to
do ExecEvalExpr on the expression tree.  Otherwise you're going to end up
reinventing most of execQual.c to cover all cases.

Right now we don't have very much infrastructure for helping FDWs push
restriction clauses over to the far end, which is what you seem to be
trying to do.  I hope that will emerge in 9.2 or 9.3.  If you want to
help you could start by looking at what the indexscan machinery does to
extract usable indexquals, because that's more or less the same problem
If you feel that's out of your league, I'd suggest not bothering with
pushing clauses across right now.

> The source does not help me much with what to do regarding the various
> fields in the struct.
> Does postgresql really fetch all columns in a subselect, or am I just
> parsing the tree in a wrong way ?

Hmm, yeah, the planner thinks it's cheaper to extract all columns than
just some of them.  This is appropriate for plain tables but I can see
that it's a bit dubious for a foreign table.  Maybe we should shut off
the use_physical_tlist optimization for foreign tables.  However, it's
not really intended that FDWs should be relying on the execution-time
targetlist to figure out which columns to pull across anyway.  The
right way to make such an optimization is for the plan-time hook to
determine which columns are needed (look at the reltargetlist) and save
that information in the plan node.

                       regards, tom lane

pgsql-general by date:

Previous
From: Martin Guether
Date:
Subject: postgres/postgis eats memory
Next
From: Eric Smith
Date:
Subject: adding a column takes FOREVER!