Thread: Updating a pre-10 partitioned table to use PG 10 partitioning

Updating a pre-10 partitioned table to use PG 10 partitioning

From
Alban Hertroys
Date:
I'm trying to update an existing table that was created in PG9.6 using
the old approach with manual inheritance and check constraints to make
use of the new approach in PG 10 using 'partitioned by', 'attach
partition', etc.

I ran into some how-to's on the internet, but they all assume you
start partitioning from a new table instead of modifying an existing
table into partitions. It seems to me that would be a fairly common
requirement, no?

I realise that I skip over the fact that I'm not attempting to
partition an unpartitioned table, but instead attempt to update the
existing partitioning to the new approach.

Currently the table in question really only has 1 partition filtered
on a type column (we plan to create 1 partition per type), but this
table is already 1.5GB on disk and has a number of indexes and views
associated to it. Creating a new table out of it and recreating all
the related stuff seems a bit roundabout...

It would be nice if I could update the table to use PG 10 partitioning
using just a simple alter table, but I can't seem to find the right
syntax.

What is a good approach here?

-- 
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.


RE: Updating a pre-10 partitioned table to use PG 10 partitioning

From
Igor Neyman
Date:

-----Original Message-----
From: Alban Hertroys [mailto:haramrae@gmail.com]
Sent: Thursday, January 11, 2018 10:41 AM
To: Postgres General <pgsql-general@postgresql.org>
Subject: Updating a pre-10 partitioned table to use PG 10 partitioning

I'm trying to update an existing table that was created in PG9.6 using the old approach with manual inheritance and check constraints to make use of the new approach in PG 10 using 'partitioned by', 'attach partition', etc.

I ran into some how-to's on the internet, but they all assume you start partitioning from a new table instead of modifying an existing table into partitions. It seems to me that would be a fairly common requirement, no?

I realise that I skip over the fact that I'm not attempting to partition an unpartitioned table, but instead attempt to update the existing partitioning to the new approach.

Currently the table in question really only has 1 partition filtered on a type column (we plan to create 1 partition per type), but this table is already 1.5GB on disk and has a number of indexes and views associated to it. Creating a new table out of it and recreating all the related stuff seems a bit roundabout...

It would be nice if I could update the table to use PG 10 partitioning using just a simple alter table, but I can't seem to find the right syntax.

What is a good approach here?

______________________________________________________________________________________________________________________

Not in one step, but it’s doable.

Here is what I do when switching from inheritance to declarative partitioning.

1. Alter all partitions to "NO INHERIT:
        
         ALTER <partition_table_name> NO INHERIT <parent_table_name>

2. Create new partitioned table  just like old “parent” table with different name using “PARTITION BY …” clause.

 

3. Drop old “parent” table.

 

4. Rename new partitioned table giving it <parent_table_name>.

 

5. Finally, “old” partitions to new partitioned table (previously “parent”):

 

      ALTER TABLE < parent_table_name > ATTACHE PARTITION <partition_table_name> FOR VALUES …

 

 

Regards,

Igor Neyman

Re: Updating a pre-10 partitioned table to use PG 10 partitioning

From
Michael Paquier
Date:
On Thu, Jan 11, 2018 at 04:40:30PM +0100, Alban Hertroys wrote:
> What is a good approach here?

By having a some point an origin and target table in the same cluster,
you would just need to define things properly. I have not checked, but
perhaps this is the kind of use cases where pg_partman
(https://github.com/keithf4/pg_partman/) can become handy? Perhaps
Keith, who maintains the tool, has some insight on the matter.
--
Michael

Attachment

Re: Updating a pre-10 partitioned table to use PG 10 partitioning

From
Keith
Date:


On Thu, Jan 11, 2018 at 8:40 PM, Michael Paquier <michael.paquier@gmail.com> wrote:
On Thu, Jan 11, 2018 at 04:40:30PM +0100, Alban Hertroys wrote:
> What is a good approach here?

By having a some point an origin and target table in the same cluster,
you would just need to define things properly. I have not checked, but
perhaps this is the kind of use cases where pg_partman
(https://github.com/keithf4/pg_partman/) can become handy? Perhaps
Keith, who maintains the tool, has some insight on the matter.
--
Michael

The method that Igor mentioned already in this email thread is currently the only way I know of to migrate an old trigger-based partition set to a new natively partitioned one in 10. You can just stick to renaming instead of doing any drops just to be safe, though. There are, as yet, no statements to turn an existing table into the parent table of a natively partitioned set in PG10. You have to create a new table, uninherit the tables from the old set (either one at a time, or all in one transaction), attach them to the new parent table, then do some table renaming to get things back to where they were before.

If you can minimize any queries on the partitioned table, the above can all be done in a single transaction so the client never knows the difference. How long it takes will be entirely dependent on any active transactions on the partition set at the time it runs since all the ALTER TABLE statements on the parent and all children all require exclusive locks. Highly recommend doing it all in one single transaction or not at all to avoid any issues. Also be aware that the native partitioning may not support everything you were doing in your old trigger based methods.

I do plan on making either a migration script or some HOWTO documentation for pg_partman. Just need to get the time to do so. Thanks for keeping me in mind!

Keith