Re: [HACKERS] Runtime Partition Pruning - Mailing list pgsql-hackers

From David Rowley
Subject Re: [HACKERS] Runtime Partition Pruning
Date
Msg-id CAKJS1f-5GeRj-g+h-BwkkqR71QvE2EdhWetRkovwKT6nEsBJMw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Runtime Partition Pruning  (Beena Emerson <memissemerson@gmail.com>)
Responses Re: [HACKERS] Runtime Partition Pruning  (Beena Emerson <memissemerson@gmail.com>)
List pgsql-hackers
On 21 December 2017 at 23:51, Beena Emerson <memissemerson@gmail.com> wrote:
> On Thu, Dec 21, 2017 at 2:31 PM, David Rowley
> <david.rowley@2ndquadrant.com> wrote:
>> On 19 December 2017 at 21:54, Beena Emerson <memissemerson@gmail.com> wrote:
>
>> The problem is down to the logic in choose_custom_plan() only choosing
>> a generic plan if the average cost of the generic plan is less than
>> the average custom plan cost. The problem is that the generic plan can
>> have many extra Append subnodes in comparison to the custom plan, all
>> of which are taken into account in the total plan cost, but these may
>> be pruned during execution. The logic in choose_custom_plan() has no
>> idea about this..
>
> I had mentioned this in the first mail on this thread that the generic
> plan is always preferred and Robert said that it is not in scope of
> this patch. Maybe we can start a new thread for this.

Sorry, I missed that. I knew you had modified choose_custom_plan()
too, but I didn't know the diagnosis of the problem had made its way
here yet.

I wonder if Robert understands the extent of the problem. The patch
will be useful to prune away partitions when the partitioned table is
on the inside of a parameterised nested loop join, but I think we'll
likely get some complaints about PREPARE statements always using a
custom plan.  It's going to make this pretty hard to write regression
tests for unless we went and invent some GUC "prefer_generic_plan", or
"generic_plan_cost_multiplier" to provide a way to coax
choose_custom_plan() into not choosing a custom plan.  I admit to not
having the answer to this but I don't think we should rule out
discussing possible fixes.

> Thank you for working on this. I  will look into this and merge with
> my current version of patch and Amit's v16 patches and post a new
> patch soon.

okay. I've just rebased mine on top of Amit's v16 now and will send
shortly. I already took some code from your v7 patch to make my v2
work. Which parts do you think still need to be merged?

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Enhance pg_stat_wal_receiver view to display connected host
Next
From: Beena Emerson
Date:
Subject: Re: [HACKERS] Runtime Partition Pruning