Thread: pgsql: Generate parallel sequential scan plans in simple cases.

Generate parallel sequential scan plans in simple cases.

Add a new flag, consider_parallel, to each RelOptInfo, indicating
whether a plan for that relation could conceivably be run inside of
a parallel worker.  Right now, we're pretty conservative: for example,
it might be possible to defer applying a parallel-restricted qual
in a worker, and later do it in the leader, but right now we just
don't try to parallelize access to that relation.  That's probably
the right decision in most cases, anyway.

Using the new flag, generate parallel sequential scan plans for plain
baserels, meaning that we now have parallel sequential scan in
PostgreSQL.  The logic here is pretty unsophisticated right now: the
costing model probably isn't right in detail, and we can't push joins
beneath Gather nodes, so the number of plans that can actually benefit
from this is pretty limited right now.  Lots more work is needed.
Nevertheless, it seems time to enable this functionality so that all
this code can actually be tested easily by users and developers.

Note that, if you wish to test this functionality, it will be
necessary to set max_parallel_degree to a value greater than the
default of 0.  Once a few more loose ends have been tidied up here, we
might want to consider changing the default value of this GUC, but
I'm leaving it alone for now.

Along the way, fix a bug in cost_gather: the previous coding thought
that a Gather node's transfer overhead should be costed on the basis of
the relation size rather than the number of tuples that actually need
to be passed off to the leader.

Patch by me, reviewed in earlier versions by Amit Kapila.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/80558c1f5aa109d08db0fbd76a6d370f900628a8

Modified Files
--------------
src/backend/nodes/outfuncs.c          |    1 +
src/backend/optimizer/path/allpaths.c |  190 ++++++++++++++++++++++++++++++++-
src/backend/optimizer/path/costsize.c |    2 +-
src/backend/optimizer/plan/planmain.c |   12 +++
src/backend/optimizer/plan/planner.c  |    9 +-
src/backend/optimizer/util/clauses.c  |  183 +++++++++++++++++++++++++------
src/backend/optimizer/util/relnode.c  |   21 ++++
src/backend/utils/cache/lsyscache.c   |   22 ++++
src/include/nodes/relation.h          |    1 +
src/include/optimizer/clauses.h       |    2 +-
src/include/utils/lsyscache.h         |    1 +
11 files changed, 400 insertions(+), 44 deletions(-)

On Wed, Nov 11, 2015 at 10:35 AM, Magnus Hagander <magnus@hagander.net> wrote:
> On Wed, Nov 11, 2015 at 4:32 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> On 11 November 2015 at 14:03, Robert Haas <rhaas@postgresql.org> wrote:
>>> Generate parallel sequential scan plans in simple cases.
>> Congratulations!
>
> +1. That's an exciting milestone!

Thank you, both of you!  I'm pretty excited.

Needless to say, there is a ton of work left.  But I hope that getting
to this point will make it easier for other people to participate in
the work, or anyway, test.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Nov 11, 2015 at 7:35 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> Congratulations!
>>
>
> +1. That's an exciting milestone!

Congratulations, Robert.


--
Peter Geoghegan

On Wed, Nov 11, 2015 at 02:03:05PM +0000, Robert Haas wrote:
> Generate parallel sequential scan plans in simple cases.

Yay!!!!!

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Wed, Nov 11, 2015 at 9:08 AM, Peter Geoghegan <pg@heroku.com> wrote:
> Congratulations, Robert.

I should certainly extend my congratulations to Amit too.

--
Peter Geoghegan

On 11/11/2015 10:54 AM, Robert Haas wrote:
> On Wed, Nov 11, 2015 at 10:35 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> On Wed, Nov 11, 2015 at 4:32 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>>> On 11 November 2015 at 14:03, Robert Haas <rhaas@postgresql.org> wrote:
>>>> Generate parallel sequential scan plans in simple cases.
>>> Congratulations!
>>
>> +1. That's an exciting milestone!
>
> Thank you, both of you!  I'm pretty excited.
>
> Needless to say, there is a ton of work left.  But I hope that getting
> to this point will make it easier for other people to participate in
> the work, or anyway, test.

This is a huge milestone! Congratulations and many thanks for making it
happen.

Joe


--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development

On Thu, Nov 12, 2015 at 4:06 AM, Joe Conway <mail@joeconway.com> wrote:
> On 11/11/2015 10:54 AM, Robert Haas wrote:
>> On Wed, Nov 11, 2015 at 10:35 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>> On Wed, Nov 11, 2015 at 4:32 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>>>> On 11 November 2015 at 14:03, Robert Haas <rhaas@postgresql.org> wrote:
>>>>> Generate parallel sequential scan plans in simple cases.
>>>> Congratulations!
>>>
>>> +1. That's an exciting milestone!
>>
>> Thank you, both of you!  I'm pretty excited.
>>
>> Needless to say, there is a ton of work left.  But I hope that getting
>> to this point will make it easier for other people to participate in
>> the work, or anyway, test.
>
> This is a huge milestone! Congratulations and many thanks for making it
> happen.

Yay.
--
Michael