Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning - Mailing list pgsql-hackers

From Amit Langote
Subject Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning
Date
Msg-id 0af7826d-3ee2-0187-f622-f86542a4f61e@lab.ntt.co.jp
Whole thread Raw
In response to Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS]path toward faster partition pruning  (David Rowley <david.rowley@2ndquadrant.com>)
Responses Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning
Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning
List pgsql-hackers
Hi David.

On 2018/01/18 12:14, David Rowley wrote:
> On 18 January 2018 at 00:13, David Rowley <david.rowley@2ndquadrant.com> wrote:
>> On 17 January 2018 at 23:48, Amit Langote <amitlangote09@gmail.com> wrote:
>>> I'm concerned that after your patch to remove
>>> match_clauses_to_partkey(), we'd be doing more work than necessary in
>>> some cases.  For example, consider the case of using run-time pruning
>>> for nested loop where the inner relation is a partitioned table.  With
>>> the old approach, get_partitions_from_clauses() would only be handed
>>> the clauses that are known to match the partition keys (which most
>>> likely is fewer than all of the query's clauses), so
>>> get_partitions_from_clauses() doesn't have to do the work of filtering
>>> non-partition clauses every time (that is, for every outer row).
>>> That's why I had decided to keep that part in the planner.
>>
>> That might be better served by splitting
>> classify_partition_bounding_keys() into separate functions, the first
>> function would be in charge of building keyclauses_all. That way the
>> remaining work during the executor would never need to match clauses
>> to a partition key as they'd be in lists dedicated to each key.
> 
> I've attached another delta against your v20 patch which does this.
> It's very rough for now and I've only checked that it passes the
> regression test so far.

Thanks!

> It will need some cleanup work, but I'd be keen to know what you think
> of the general idea.

This one looks in a much better shape.

> I've not fully worked out how run-time pruning
> will use this as it'll need another version of
> get_partitions_from_clauses but passes in a PartScanClauseInfo
> instead, and does not call extract_partition_key_clauses. That area
> probably  needs some shuffling around so that does not end up a big
> copy and paste of all that new logic.
So, I've been assuming that the planner changes in the run-time pruning
patch have to do with selecting clauses (restriction clauses not
containing Consts and/or join clauses) to be passed to the executor by
recording them in the Append node.  Will they be selected by the planner
calling into partition.c?

Meanwhile, here is a revised version (v21) that incorporates your changes.
 I added you as the author in 0002 and 0005 patches.  I guess a v22 will
have to follow very soon...

Thanks,
Amit

Attachment

pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Next
From: hubert depesz lubaczewski
Date:
Subject: Re: Is there a "right" way to test if a database is empty?