Re: Asynchronous Append on postgres_fdw nodes. - Mailing list pgsql-hackers

From Andrey V. Lepikhov
Subject Re: Asynchronous Append on postgres_fdw nodes.
Date
Msg-id 581520e2-1475-c614-b879-98b350220508@postgrespro.ru
Whole thread Raw
In response to Re: Asynchronous Append on postgres_fdw nodes.  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
Responses Re: Asynchronous Append on postgres_fdw nodes.
List pgsql-hackers
On 6/16/20 1:30 PM, Kyotaro Horiguchi wrote:
> They return 25056 rows, which is far more than 9741 rows. So remote
> join won.
> 
> Of course the number of returning rows is not the only factor of the
> cost change but is the most significant factor in this case.
> 
Thanks for the attention.
I see one slight flaw of this approach to asynchronous append:
AsyncAppend works only for ForeignScan subplans. if we have 
PartialAggregate, Join or another more complicated subplan, we can't use 
asynchronous machinery.
It may lead to a situation than small difference in a filter constant 
can cause a big difference in execution time.
I imagine an Append node, that can switch current subplan from time to 
time and all ForeignScan nodes of the overall plan are added to one 
queue. The scan buffer can be larger than a cursor fetch size and each 
IterateForeignScan() call can induce asynchronous scan of another 
ForeignScan node if buffer is not full.
But these are only thoughts, not an proposal. I have no questions to 
your patch right now.

-- 
Andrey Lepikhov
Postgres Professional
https://postgrespro.com



pgsql-hackers by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: language cleanups in code and docs
Next
From: Laurenz Albe
Date:
Subject: Re: language cleanups in code and docs