Re: Async execution of postgres_fdw. - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Async execution of postgres_fdw.
Date
Msg-id 1608.1431145073@sss.pgh.pa.us
Whole thread Raw
In response to Re: Async execution of postgres_fdw.  (Stephen Frost <sfrost@snowman.net>)
Responses Re: Async execution of postgres_fdw.
List pgsql-hackers
Stephen Frost <sfrost@snowman.net> writes:
> I'm all for improving performance of postgres_fdw and would like to see
> us support sending queries off to be worked asyncronously, but starting
> execution on the remote server during ExecInitNode is against the
> documentated FDW API spec.  I discussed exactly this issue over a year
> ago here:

> http://www.postgresql.org/message-id/20131104032604.GB2706@tamriel.snowman.net

> Sadly, there weren't any direct responses to that email, but I do recall
> having a discussion on another thread (or in person?) with Tom where we
> ended up agreeing that we can't simply remove that requirement from the
> docs or the API.

Yeah.  There are at least a couple of reasons why not:

* ExecInitNode only creates the runtime data structures, it should not
begin execution.  It's possible for example that the scan will never be
iterated at all; say it's on the inside of a nestloop and the outer
relation turns out to be empty.  It's not apparent why starting the remote
query a few microseconds sooner is worth the risk of demanding useless
computation.

* If the scan is parameterized (again, it's on the inside of a nestloop,
and the outer relation is supplying join key values), those parameter
values are simply not available at ExecInitNode time.


Also, so far as a quick review of the actual patch goes, I would really
like to see this lose the "PFC" wrapper layer, which accounts for 95% of
the code churn in the patch and doesn't seem to add any actual value.
What it does add is unchecked malloc failure conditions.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Stas Kelvich
Date:
Subject: Re: Cube extension kNN support
Next
From: Abhijit Menon-Sen
Date:
Subject: Re: a fast bloat measurement tool (was Re: Measuring relation free space)