Thread: FDW / list of needed columns, WHERE conditions (in PlanForeignScan)

Hi,

I'm writing my first FDW, and I need to get the list of columns I
actually need to fetch when planning the query. I do want to fetch only
the columns that are actually needed, not all of them.

Initially I've used RelOptInfo->reltargetlist but it seems it does not
cover cases like
  SELECT 1 FROM FDW WHERE column_a = 1

and so on. Even RelOptInfo->attr_needed seems not to work as this
returns "not needed" for all columns:
   for (i = baserel->min_attr; i <= baserel->max_attr; i++) {       if (baserel->attr_needed[i-baserel->min_attr] !=
NULL)          elog(WARNING, "attr %d needed", i);       else           elog(WARNING, "attr %d not needed", i);   }
 

Where to get this info?

Also, I'd like to apply as much restrictions as possible when executing
the plan. I see there's PlanState->qual, described as "implicitly-ANDed
qual conditions", which I assume is what I need. But this seems to be
available only in BeginForeignScan (as ss.ps.qual in ForeignScanState),
not in planning which is the place where I need to compute estimates
etc. Where do I get this, when planning the query?

kind regards
Tomas

Tomas Vondra <tv@fuzzy.cz> writes:
> I'm writing my first FDW, and I need to get the list of columns I
> actually need to fetch when planning the query. I do want to fetch only
> the columns that are actually needed, not all of them.

reltargetlist and attr_needed only tell you about columns the scan has
to *output* (ie, they are used in join conditions or the final result).
Vars that are only mentioned in baserestrict conditions aren't
included.  So you'd need to do something like pull_varattnos on the
baserestrictinfo list and union that with attr_needed.

> Also, I'd like to apply as much restrictions as possible when executing
> the plan. I see there's PlanState->qual, described as "implicitly-ANDed
> qual conditions", which I assume is what I need. But this seems to be
> available only in BeginForeignScan (as ss.ps.qual in ForeignScanState),
> not in planning which is the place where I need to compute estimates
> etc. Where do I get this, when planning the query?

Same answer, rel->baserestrictinfo.
        regards, tom lane

On 28.5.2012 23:20, Tom Lane wrote:
> Tomas Vondra <tv@fuzzy.cz> writes:
>> I'm writing my first FDW, and I need to get the list of columns I
>> actually need to fetch when planning the query. I do want to fetch only
>> the columns that are actually needed, not all of them.
> 
> reltargetlist and attr_needed only tell you about columns the scan has
> to *output* (ie, they are used in join conditions or the final result).
> Vars that are only mentioned in baserestrict conditions aren't
> included.  So you'd need to do something like pull_varattnos on the
> baserestrictinfo list and union that with attr_needed.
> 
>> Also, I'd like to apply as much restrictions as possible when executing
>> the plan. I see there's PlanState->qual, described as "implicitly-ANDed
>> qual conditions", which I assume is what I need. But this seems to be
>> available only in BeginForeignScan (as ss.ps.qual in ForeignScanState),
>> not in planning which is the place where I need to compute estimates
>> etc. Where do I get this, when planning the query?
> 
> Same answer, rel->baserestrictinfo.
> 
>             regards, tom lane
> 

Thanks for the explanation, but it leads me to another question ...

Do I need to return only the data in rel->reltargetlist, or should I
return all the columns (including those in rel->baserestrictinfo)?

Because AFAIK PostgreSQL will recheck the conditions on the data
returned from FDW and if I only return the attributes from the
reltargetlist, the conditions will evaluate to FALSE. Can I prevent the
recheck somehow e.g. by deleting it from baserestrictinfo?

Also, how is baserestrictinfo related to PlanState.quals? Actually what
are 'qual conditions'? I see 'qual' is a list of conditions connected by
AND, but I'm not sure what a 'qual' means. Is that equality?

kind regards
Tomas

Tomas Vondra <tv@fuzzy.cz> writes:
> Do I need to return only the data in rel->reltargetlist, or should I
> return all the columns (including those in rel->baserestrictinfo)?

> Because AFAIK PostgreSQL will recheck the conditions on the data
> returned from FDW and if I only return the attributes from the
> reltargetlist, the conditions will evaluate to FALSE. Can I prevent the
> recheck somehow e.g. by deleting it from baserestrictinfo?

In 9.1, the FDW doesn't have any control over that, so yeah you have to
include all those columns in the constructed scan tuples.  We rejiggered
this in 9.2 so that the FDW could control which baserestrictinfo quals
are evaluated that way.  (Of course, it's up to you to be sure you do
somehow check any conditions you remove from the qual list.)

> Also, how is baserestrictinfo related to PlanState.quals? Actually what
> are 'qual conditions'? I see 'qual' is a list of conditions connected by
> AND, but I'm not sure what a 'qual' means. Is that equality?

It's just planner slang for a WHERE condition.  The List structure
corresponds to top-level AND structure in WHERE, but otherwise the
elements correspond directly to WHERE expressions.
        regards, tom lane