On Fri, Feb 28, 2014 at 10:36 AM, Stephen Frost <sfrost@snowman.net> wrote:
> * Kouhei Kaigai (kaigai@ak.jp.nec.com) wrote:
>> * Stephen Frost (sfrost@snowman.net) wrote:
>> > I don't see how you can be when there hasn't been any discussion that I've
>> > seen about how parallel query execution is going to change things for us.
>> >
>> If parallel query execution changes whole of the structure of plan nodes,
>> it will also affect to the interface of custom-scan because it is a thin-
>> abstraction of plan-node. However, if parallel execution feature is
>> implemented as one of new plan node in addition to existing one, I cannot
>> imagine a scenario that affects to the structure of another plan node.
>
> Let's just say that I have doubts that we'll be able to implement
> parallel execution *without* changing the plan node interface in some
> way which will require, hopefully minor, changes to all of the nodes.
> The issue is that even a minor change would break the custom-scan API
> and we'd immediately be in the boat of dealing with complaints regarding
> backwards compatibility. Perhaps we can hand-wave that, and we've had
> some success changing hook APIs between major releases, but such changes
> may also be in ways which wouldn't break in obvious ways or even
> possibly be changes which have to be introduced into back-branches.
> Parallel query is going to be brand-new real soon and it's reasonable to
> think we'll need to make bug-fix changes to it after it's out which
> might even involve changes to the API which is developed for it.
For what it's worth, and I can't claim to have all the answers here,
this doesn't match my expectation. I think we'll do two kinds of
parallelism. One will be parallelism within nodes, like parallel sort
or parallel seqscan. Any node we parallelize this way is likely to be
heavily rewritten, or else to get a sister that looks quite different
from the original. The other kind of parallelism will involve pushing
a whole subtree of the plan into a different node. In this case we'll
need to pass data between nodes in some different way (this was one of
the major reasons I designed the shm_mq stuff) but the nodes
themselves should change little if at all.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
