Re: [HACKERS] Adding support for Default partition in partitioning - Mailing list pgsql-hackers

From Jeevan Ladhe
Subject Re: [HACKERS] Adding support for Default partition in partitioning
Date
Msg-id CAOgcT0OARciE2X+U0rjSKp9VuC279dYcCGkc3nCWKhHQ1_m2rw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Adding support for Default partition in partitioning  (Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>)
Responses Re: [HACKERS] Adding support for Default partition in partitioning
Re: [HACKERS] Adding support for Default partition in partitioning
List pgsql-hackers
Hi,

I have worked further on V21 patch set, rebased it on latest master commit,
addressed the comments given by Robert, Ashutosh and others.

The attached tar has a series of 7 patches.
Here is a brief of these 7 patches:

0001:
Refactoring existing ATExecAttachPartition  code so that it can be used for
default partitioning as well

0002:
This patch teaches the partitioning code to handle the NIL returned by
get_qual_for_list().
This is needed because a default partition will not have any constraints in case
it is the only partition of its parent.

0003:
Support for default partition with the restriction of preventing addition of any
new partition after default partition.

0004:
Store the default partition OID in pg_partition_table, this will help us to
retrieve the OID of default relation when we don't have the relation cache
available. This was also suggested by Amit Langote here[1].

0005:
Extend default partitioning support to allow addition of new partitions.

0006:
Extend default partitioning validation code to reuse the refactored code in
patch 0001. 

0007:
This patch introduces code to check if the scanning of default partition child
can be skipped if it's constraints are proven.

TODO:
Add documentation.
Merge default range partitioning patch.

Regards,
Jeevan Ladhe

On Fri, Jun 30, 2017 at 5:48 PM, Jeevan Ladhe <jeevan.ladhe@enterprisedb.com> wrote:
Hi,

On Mon, Jun 19, 2017 at 12:34 PM, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/06/16 14:16, Ashutosh Bapat wrote:
> On Fri, Jun 16, 2017 at 12:48 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Thu, Jun 15, 2017 at 12:54 PM, Ashutosh Bapat
>> <ashutosh.bapat@enterprisedb.com> wrote:
>>> Some more comments on the latest set of patches.

>> or looking up the OID in the
>> relcache multiple times.
>
> I am not able to understand this in the context of default partition.
> After that nobody else is going to change its partitions and their
> bounds (since both of those require heap_open on parent which would be
> stuck on the lock we hold.). So, we have to check only once if the
> table has a default partition. If it doesn't, it's not going to
> acquire one unless we release the lock on the parent i.e at the end of
> transaction. If it has one, it's not going to get dropped till the end
> of the transaction for the same reason. I don't see where we are
> looking up OIDs multiple times.

Without heap_opening the parent, the only way is to look up parentOid's
children in pg_inherits and for each child looking up its pg_class tuple
in the syscache to see if its relpartbound indicates that it's a default
partition.  That seems like it won't be inexpensive either.

It would be nice if could get that information (that is - is a given
relation being heap_drop_with_catalog'd a partition of the parent that
happens to have default partition) in less number of steps than that.
Having that information in relcache is one way, but as mentioned, that
turns out be expensive.

Has anyone considered the idea of putting the default partition OID in the
pg_partitioned_table catalog?  Looking the above information up would
amount to one syscache lookup.  Default partition seems to be special
enough object to receive a place in the pg_partitioned_table tuple of the
parent.  Thoughts?
 
I liked this suggestion. Having an entry in pg_partitioned_table would avoid
both expensive methods, i.e. 1. opening the parent or 2. lookup for
each of the children first in pg_inherits and then its corresponding entry in
pg_class.
Unless anybody has any other suggestions/comments here, I am going to
implement this suggestion.

Thanks,
Jeevan Ladhe

Attachment

pgsql-hackers by date:

Previous
From: Peter Geoghegan
Date:
Subject: Re: [HACKERS] [WIP] Zipfian distribution in pgbench
Next
From: Peter Geoghegan
Date:
Subject: Re: [HACKERS] [WIP] Zipfian distribution in pgbench