Re: Moving SS_finalize_plan processing to the end of planning - Mailing list pgsql-hackers

From David Rowley
Subject Re: Moving SS_finalize_plan processing to the end of planning
Date
Msg-id CAKJS1f9s+iHKkMjvwPyAyPa_a+3=QTHu2Sn2VB0EiL2=5T557A@mail.gmail.com
Whole thread Raw
In response to Moving SS_finalize_plan processing to the end of planning  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Moving SS_finalize_plan processing to the end of planning
List pgsql-hackers
On 10 August 2015 at 07:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I've started to work on path-ification of the upper planner (finally),
and since that's going to be a large patch in any case, I've been looking
for pieces that could be bitten off and done separately.  One such piece
is the fact that SS_finalize_plan (computation of extParams/allParams)
currently gets run at the end of subquery_planner; as long as that's true,
we cannot have subquery_planner returning paths rather than concrete
plans.  The attached patch moves that processing to occur just before
set_plan_references is run.

Basically what this patch does is to divide what had been done in
SS_finalize_plan into three steps:

* SS_identify_outer_params determines which outer-query-level Params will
be available for the current query level to use, and saves that aside in
a new PlannerInfo field root->outer_params.  This processing turns out
to be the only reason that SS_finalize_plan had to be called in
subquery_planner: we have to capture this data before exiting
subquery_planner because the upper levels' plan_params lists may change
as they plan other subplans.

* SS_attach_initplans does the work of attaching initplans to the parent
plan node and adjusting the parent's cost estimate accordingly.

* SS_finalize_plan now *only* does extParam/allParam calculation.

I had to change things around a bit in planagg.c (which was already a
hack, and the law of conservation of cruft applies).  Otherwise it's
pretty straightforward and removes some existing hacks.  One notable
point is that there's no longer an assumption within SS_finalize_plan
that initPlans can only appear in the top plan node.

Any objections?

Great! I'm very interested in this work, specifically around the grouping_planner() changes too.

I've looked over the patch and from what I understand it seems like a good solid step in the right direction.

I was digging around the grouping_planner() last week with the intentions of making some changes there to allow GROUP BY before join, but in the end decided to stay well clear of this area until this pathification is done. So far I've managed to keep my changes away from the upper planner and I've added "GroupingPath" types, which from what I can predict of what you'll be making changes to, I think you'll also need to have grouping_planner() return a few variations of "GroupingPath" to allow the paths list to be passed up to subquery_planner() and on up to functions like recurse_set_operations() so that they have the option of choosing GroupAggregate / MergeAppend to implement UNION.

If I'm right on this, then maybe there's a few things you can copy and paste from the patch I posted here: http://www.postgresql.org/message-id/CAKJS1f-sEcm=gTfS-DqjsOcsZ-vLhrP_hSRNtJjq-S7Egn8Rqw@mail.gmail.com  specifically around create_grouping_path()?

Kind Regards

David Rowley

--
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
 

pgsql-hackers by date:

Previous
From: Shay Rojansky
Date:
Subject: Re: statement_timeout affects query results fetching?
Next
From: Petr Jelinek
Date:
Subject: Re: WIP: Rework access method interface