Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning - Mailing list pgsql-hackers

From Amit Langote
Subject Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning
Date
Msg-id 29e58ace-d7e2-7945-bd42-2f4769ceb6db@lab.ntt.co.jp
Whole thread Raw
In response to Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning  (David Rowley <david.rowley@2ndquadrant.com>)
Responses Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning
List pgsql-hackers
Hi David.

On 2018/01/23 15:44, David Rowley wrote:
> On 19 January 2018 at 04:03, David Rowley <david.rowley@2ndquadrant.com> wrote:
>> On 18 January 2018 at 23:56, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>>> So, I've been assuming that the planner changes in the run-time pruning
>>> patch have to do with selecting clauses (restriction clauses not
>>> containing Consts and/or join clauses) to be passed to the executor by
>>> recording them in the Append node.  Will they be selected by the planner
>>> calling into partition.c?
>>
>> I had thought so. I only have a rough idea in my head and that's that
>> the PartitionPruneInfo struct that I wrote for the run-time pruning
>> patch would have the clause List replaced with this new
>> PartScanClauseInfo struct (which likely means it needs to go into
>> primnodes.h), this struct would contain all the partition pruning
>> clauses in a more structured form so that no matching of quals to the
>> partition key wouldn't be required during execution. The idea is that
>> we'd just need to call; remove_redundant_clauses,
>> extract_bounding_datums and get_partitions_for_keys. I think this is
>> the bare minimum of work that can be done in execution since we can't
>> remove the redundant clauses until we know the values of the Params.
>>
>> Likely this means there will need to be 2 functions externally
>> accessible for this in partition.c. I'm not sure exactly how best to
>> do this. Maybe it's fine just to have allpaths.c call
>> extract_partition_key_clauses to generate the PartScanClauseInfo then
>> call some version of get_partitions_from_clauses which does do the
>> extract_partition_key_clauses duties. This is made more complex by the
>> code that handles adding the default partition bound to the quals. I'm
>> not yet sure where that should live.
>>
>> I've also been thinking of having some sort of PartitionPruneContext
>> struct that we can pass around the functions. Runtime pruning had to
>> add structs which store the Param values to various functions which I
>> didn't like. It would be good to just add those to the context and
>> have them passed down without having to bloat the parameters in the
>> functions. I might try and do that tomorrow too. This should make the
>> footprint of the runtime pruning patch is a bit smaller.
> 
> Attached is what I had in mind about how to do this.

Thanks for the delta patch.  I will start looking at it tomorrow.

Regards,
Amit



pgsql-hackers by date:

Previous
From: Simon Riggs
Date:
Subject: Re: Rangejoin rebased
Next
From: Константин Евтеев
Date:
Subject: Invalidation pg catalog cache in trigger functions