Re: Declarative partitioning - another take - Mailing list pgsql-hackers

From Ashutosh Bapat
Subject Re: Declarative partitioning - another take
Date
Msg-id CAFjFpRcvF0HjMvSMTTbGOwXLgrgCoiSDV42odWJWnF9+bA76pg@mail.gmail.com
Whole thread Raw
In response to Re: Declarative partitioning - another take  (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>)
Responses Re: Declarative partitioning - another take  (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>)
List pgsql-hackers


On Thu, Aug 25, 2016 at 12:22 PM, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2016/08/22 13:51, Ashutosh Bapat wrote:
> The parent-child relationship of multi-level partitioned tables is not
> retained when creating the AppendRelInfo nodes. We create RelOptInfo nodes
> for all the leaf and intermediate tables. The AppendRelInfo nodes created
> for these RelOptInfos set the topmost table as the parent of all the leaf
> and child tables. Since partitioning scheme/s at each level is/are
> associated with the parent/s at that level, we loose information about the
> immediate parents and thus it becomes difficult to identify which leaf node
> falls where in the partition hierarchy. This stops us from doing any
> lump-sum partition pruning where we can eliminate all the partitions under
> a given parent-partition if that parent-partition gets pruned. It also
> restricts partition-wise join technique from being applied to partial
> partition hierarchy when the whole partitioning scheme of joining tables
> does not match. Maintaining a RelOptInfo hierarchy should not create
> corresponding Append (all kinds) plan hierarchy since
> accumulate_append_subpath() flattens any such hierarchy while creating
> paths. Can you please consider this point in your upcoming patch?

I agree.  So there seem to be two things here:  a) when expanding a
partitioned table inheritance set, do it recursively such that resulting
AppendRelInfos preserve *immediate* parent-child relationship info. 

Right.
 
b)
when accumulating append subpaths, do not flatten a subpath that is itself
an append when ((AppendPath *) subpath)->path.parent is a RelOptInfo with
non-NULL partitioning info.Is the latter somehow necessary for
pairwise-join considerations?

I don't think you need to do anything in the path creation code for this. As is it flattens all AppendPath hierarchies whether for partitioning or inheritance or subqueries. We should leave it as it is.



I think I can manage to squeeze in (a) in the next version patch and will
also start working on (b), mainly the part about RelOptInfo getting some
partitioning info.

I am fine with b, where you would include some partitioning information in RelOptInfo. But you don't need to do what you said in (b) above.

In a private conversation Robert Haas suggested a way slightly different than what my patch for partition-wise join does. He suggested that the partitioning schemes i.e strategy, number of partitions and bounds of the partitioned elations involved in the query should be stored in PlannerInfo in the form of a list. Each partitioning scheme is annotated with the relids of the partitioned relations. RelOptInfo of the partitioned relation will point to the partitioning scheme in PlannerInfo. Along-with that each RelOptInfo will need to store partition keys for corresponding relation. This simplifies matching the partitioning schemes of the joining relations. Also it reduces the number of copies of partition bounds floating around as we expect that a query will involve multiple partitioned tables following similar partitioning schemes. May be you want to consider this idea while working on (b).


Thanks,
Amit





--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

pgsql-hackers by date:

Previous
From: Amit Langote
Date:
Subject: Re: Declarative partitioning - another take
Next
From: Petr Jelinek
Date:
Subject: Re: Logical Replication WIP