Re: [HACKERS] Gather Merge - Mailing list pgsql-hackers
| From | Rushabh Lathia |
|---|---|
| Subject | Re: [HACKERS] Gather Merge |
| Date | |
| Msg-id | CAGPqQf1r7XjhPPDqnuEbQgTnk2UXmizqjnkHTT2uUAgJnLyrzA@mail.gmail.com Whole thread Raw |
| In response to | Re: [HACKERS] Gather Merge (Rushabh Lathia <rushabh.lathia@gmail.com>) |
| Responses |
Re: [HACKERS] Gather Merge
(Robert Haas <robertmhaas@gmail.com>)
|
| List | pgsql-hackers |
Error coming from create_gather_merge_plan() from below condition:
if (memcmp(sortColIdx, gm_plan->sortColIdx,
numsortkeys * sizeof(AttrNumber)) != 0)
elog(ERROR, "GatherMerge child's targetlist doesn't match GatherMerge");
Above condition checks the sort column numbers explicitly, to ensure the tlists
really do match up. This been copied from the create_merge_append_plan(). Now
this make sense as for MergeAppend as its not projection capable plan (see
is_projection_capable_plan()). But like Gather, GatherMerge is the projection
capable and its target list can be different from the subplan, so I don't think this
condition make sense for the GatherMerge.
Here is the some one the debugging info, through which I was able to reach
to this conclusion:
- targetlist for the GatherMerge and subpath is same during create_gather_merge_path().
- targetlist for the GatherMerge is getting changed into create_gather_merge_plan().
postgres=# explain (analyze, verbose) select t2.j from t1 JOIN t2 ON t1.k=t2.k where t1.i=1 order by t1.j desc;
NOTICE: path parthtarget: {PATHTARGET :exprs ({VAR :varno 2 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 2 :varoattno 2 :location 34} {VAR :varno 1 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 2 :location 90}) :sortgrouprefs 0 1 :cost.startup 0.00 :cost.per_tuple 0.00 :width 8}
NOTICE: subpath parthtarget: {PATHTARGET :exprs ({VAR :varno 1 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 2 :location 90} {VAR :varno 2 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 2 :varoattno 2 :location 34}) :cost.startup 0.00 :cost.per_tuple 0.00 :width 8}
- Attached memory watch point and found that target list for GatherMerge is getting
changed into groupping_planner() -> apply_projection_to_path().
PFA patch to fix this issue.if (memcmp(sortColIdx, gm_plan->sortColIdx,
numsortkeys * sizeof(AttrNumber)) != 0)
elog(ERROR, "GatherMerge child's targetlist doesn't match GatherMerge");
Above condition checks the sort column numbers explicitly, to ensure the tlists
really do match up. This been copied from the create_merge_append_plan(). Now
this make sense as for MergeAppend as its not projection capable plan (see
is_projection_capable_plan()). But like Gather, GatherMerge is the projection
capable and its target list can be different from the subplan, so I don't think this
condition make sense for the GatherMerge.
Here is the some one the debugging info, through which I was able to reach
to this conclusion:
- targetlist for the GatherMerge and subpath is same during create_gather_merge_path().
- targetlist for the GatherMerge is getting changed into create_gather_merge_plan().
postgres=# explain (analyze, verbose) select t2.j from t1 JOIN t2 ON t1.k=t2.k where t1.i=1 order by t1.j desc;
NOTICE: path parthtarget: {PATHTARGET :exprs ({VAR :varno 2 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 2 :varoattno 2 :location 34} {VAR :varno 1 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 2 :location 90}) :sortgrouprefs 0 1 :cost.startup 0.00 :cost.per_tuple 0.00 :width 8}
NOTICE: subpath parthtarget: {PATHTARGET :exprs ({VAR :varno 1 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 2 :location 90} {VAR :varno 2 :varattno 2 :vartype 23 :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 2 :varoattno 2 :location 34}) :cost.startup 0.00 :cost.per_tuple 0.00 :width 8}
- Attached memory watch point and found that target list for GatherMerge is getting
changed into groupping_planner() -> apply_projection_to_path().
On Fri, Mar 10, 2017 at 4:12 PM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
On Fri, Mar 10, 2017 at 4:09 PM, Kuntal Ghosh <kuntalghosh.2007@gmail.com> wrote:On Fri, Mar 10, 2017 at 3:04 PM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:
>
>
> On Fri, Mar 10, 2017 at 2:42 PM, Andreas Joseph Krogh <andreas@visena.com>
> wrote:
>>
>> På fredag 10. mars 2017 kl. 10:09:22, skrev Rushabh Lathia
>> <rushabh.lathia@gmail.com>:
>>
>>
>>
>> On Fri, Mar 10, 2017 at 2:33 PM, Andreas Joseph Krogh <andreas@visena.com>
>> wrote:
>>>
>>> [...]
>>> The execution-plan seems (unsurprisingly) to depend on data-distribution,
>>> so is there a way I can force a GatherMerge?
>>
>>
>> Not directly. GatherMerge cost is mainly depend on parallel_setup_cost,
>> parallel_tuple_cost and cpu_operator_cost. May be you can force this
>> by setting this cost low enough. Or another way to force is by disable the
>> other plans.
>>
>> What plan you are getting now? You not seeing the below error ?
>>
>> ERROR: GatherMerge child's targetlist doesn't match GatherMerge
>>
>>
>> I'm seeing the same error, it's just that for reproducing it I'd rather
>> not copy my whole dataset.
>
>
> Can you share me a schema information, I will try to reproduce at my side?
I'm able to reproduce the error. I've attached the dump file and a
script to reproduce it.
The following query executes successfully.
postgres=# explain select t1.* from t1 JOIN t2 ON t1.k=t2.k where
t1.i=1 order by t1.j desc;
QUERY PLAN
------------------------------------------------------------ ------------------------------ -------------
Gather Merge (cost=0.58..243.02 rows=943 width=12)
Workers Planned: 1
-> Nested Loop (cost=0.57..235.94 rows=555 width=12)
-> Parallel Index Scan Backward using idx_t1_i_j on t1
(cost=0.29..14.33 rows=603 width=12)
Index Cond: (i = 1)
-> Index Only Scan using idx_t2_k on t2 (cost=0.29..0.34
rows=3 width=4)
Index Cond: (k = t1.k)
(7 rows)
Whereas, If columns from t2 is projected, it throws the same error.
postgres=# explain select t2.* from t1 JOIN t2 ON t1.k=t2.k where
t1.i=1 order by t1.j desc;
ERROR: GatherMerge child's targetlist doesn't match GatherMergeThanks Kuntal.I am able to reproduce the issue with the shared script. I will look intothis now.--Rushabh Lathia
--
Rushabh Lathia
Attachment
pgsql-hackers by date: