Re: Custom Scan APIs (Re: Custom Plan node) - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: Custom Scan APIs (Re: Custom Plan node)
Date
Msg-id 20140302020428.GI12995@tamriel.snowman.net
Whole thread Raw
In response to Re: Custom Scan APIs (Re: Custom Plan node)  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Custom Scan APIs (Re: Custom Plan node)  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
* Robert Haas (robertmhaas@gmail.com) wrote:
> I don't see that parallelizing Append is any easier than any other
> problem in this space.  There's no parallel I/O facility, so you need
> a background worker per append branch to wait on I/O.  And you have
> all the problems of making sure that the workers have the same
> snapshot, making sure they don't self-deadlock, etc. that you have for
> any other case.

Erm, my thought was to use a select() loop which sends out I/O requests
and then loops around waiting to see who finishes it.  It doesn't
parallelize the CPU cost of getting the rows back to the caller, but
it'd at least parallelize the I/O, and if what's underneath is actually
a remote FDW running a complex query (because the other side is actually
a view), it would be a massive win to have all the remote FDWs executing
concurrently instead of serially as we have today.
Thanks,
    Stephen

pgsql-hackers by date:

Previous
From: Kohei KaiGai
Date:
Subject: Re: Custom Scan APIs (Re: Custom Plan node)
Next
From: Stephen Frost
Date:
Subject: Re: Custom Scan APIs (Re: Custom Plan node)