Thread: Parallel queries

<p><font size="2">All, </font><p><font size="2">How hard would it be to get a back end to hand off parts of a query
treeto other back ends (which it created), and then collate the results, and take it from there?  Obviously, it would
onlydo this under certain conditions, specifically, if it was compiled in to the server, and the query tree in question
hada particular profile, i.e.: lent itself to parallel processing.  But surely it couldn't be that difficult to get a
backend to hand off, say, a sub-select sub-tree, or UNION sub-trees.  Under heavy loads with the right hardware, this
couldgain significant performance improvement.</font><br /><p><font size="2">Michael Ansley</font><code><font
size="3"><br/><br /> **********************************************************************<br /> This email and any
filestransmitted with it are confidential and<br /> intended solely for the use of the individual or entity to whom
they<br/> are addressed. If you have received this email in error please notify<br /> Nick West - Global Infrastructure
Manager.<br/><br /> This footnote also confirms that this email message has been swept by<br /> MIMEsweeper for the
presenceof computer viruses.<br /><br /> www.mimesweeper.com<br />
**********************************************************************<br/></font></code> 

> How hard would it be to get a back end to hand off parts of a query
> tree to other back ends (which it created), and then collate the
> results, and take it from there?  Obviously, it would only do this
> under certain conditions, specifically, if it was compiled in to the
> server, and the query tree in question had a particular profile, i.e.:
> lent itself to parallel processing.  But surely it couldn't be that
> difficult to get a back end to hand off, say, a sub-select sub-tree,
> or UNION sub-trees.  Under heavy loads with the right hardware, this
> could gain significant performance improvement.

Right. This would help speed up some single-client queries on SMP
machines (we already scale will with multiple clients on SMP machines).
This could perhaps be solved by threading each client backend, or, as
you suggest, by spawning entirely new processes (which would easily ??
generalize into distributed database support). Bring it up after 7.1 is
released (though there may be enough hoops to jump through that it would
not be in 7.2; not certain yet).

All imho of course, though we've had a few discussions off-list to help
get us thinking about it for planning purposes.

I would assume that it will take a few folks working together to get
this to fly, but feel free to give it a go on your own.
                     - Thomas