On Tue, Jan 26, 2016 at 1:30 AM, Kouhei Kaigai <kaigai@ak.jp.nec.com> wrote:
> What enhancement will be necessary to implement similar feature of
> partial seq-scan using custom-scan interface?
>
> It seems to me callbacks on the three points below are needed.
> * ExecParallelEstimate
> * ExecParallelInitializeDSM
> * ExecParallelInitializeWorker
>
> Anything else?
> Does ForeignScan also need equivalent enhancement?
For postgres_fdw, running the query from a parallel worker would
change the transaction semantics. Suppose you begin a transaction,
UPDATE data on the foreign server, and then run a parallel query. If
the leader performs the ForeignScan it will see the uncommitted
UPDATE, but a worker would have to make its own connection which not
be part of the same transaction and which would therefore not see the
update. That's a problem.
Also, for postgres_fdw, and many other FDWs I suspect, the assumption
is that most of the work is being done on the remote side, so doing
the work in a parallel worker doesn't seem super interesting. Instead
of incurring transfer costs to move the data from remote to local, we
incur two sets of transfer costs: first remote to local, then worker
to leader. Ouch. I think a more promising line of inquiry is to try
to provide asynchronous execution when we have something like:
Append
-> Foreign Scan
-> Foreign Scan
...so that we can return a row from whichever Foreign Scan receives
data back from the remote server first.
So it's not impossible that an FDW author could want this, but mostly
probably not. I think.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
