Re: [HACKERS] Documentation improvements for partitioning - Mailing list pgsql-hackers
From | Petr Jelinek |
---|---|
Subject | Re: [HACKERS] Documentation improvements for partitioning |
Date | |
Msg-id | c3c15847-27ea-0a1c-2a9a-ec5adf2d7b41@2ndquadrant.com Whole thread Raw |
In response to | Re: [HACKERS] Documentation improvements for partitioning (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>) |
List | pgsql-hackers |
On 27/02/17 07:59, Amit Langote wrote: > On 2017/02/27 3:18, Petr Jelinek wrote: >> On 24/02/17 07:15, Robert Haas wrote: >>> On Fri, Feb 24, 2017 at 9:53 AM, Simon Riggs <simon@2ndquadrant.com> wrote: >>>> The good news is that logical replication DOES work with partitioning, >>>> but only for a Publication with PUBLISH INSERT, pushing from a normal >>>> table to a partitioned one. Which is useful enough for first release. >>>> >>>> The work on having UPDATE work between partitions can be used to make >>>> updates and deletes work in logical replication. That might yet be >>>> made to work in this release, and I think we should look into it, but >>>> I think it can be left til next release if we try. >>> >>> Are you speaking of the case where you want to replicate from an >>> unpartitioned table to a partitioned table? I would imagine that if >>> you are replicating between two identical partitioning hierarchies, it >>> would just work. (If not, that's probably a bug, though I don't know >>> whose bug.) >>> >> >> Yes same hierarchies will work but mainly because one has to add >> partitions themselves to publication currently. I guess that's the >> limitation we might have to live with in 10 as adding the whole >> partitioned table should probably work for different hierarchies when we >> enable it and I am not quite sure that's doable before start of the CF >> at this point. > > If and when we add support to add partitioned tables to publications, I > think it will work by recursing to include the partitions to the same > publication (I see that OpenTableList() in publicationcmds.c calls > find_all_inheritors if recursion is requested by not specifying ONLY). > When a subscription replicates from this publication, it's going to match > changes for individual leaf partitions, not the root parent table. IOW, > none of the changes applied to a partitioned table are replicated as > changes to the table itself. So, it seems that adding a partitioned table > to a publication or subscription would simply be a *shorthand* for adding > all the (leaf) partitions that will actually emit and receive changes. This was my first thought as well. However we need to also take into account the use-case with different partitioning topology on publisher and subscriber (at least in a sense that the initial design will not paint us in the corner). Now I see two ways of achieving this. a) Either going with more or less what you wrote above and in the future have some smarts where we can specify that it should replicate with different name. We'll eventually want to be able to replicate table to differently named table anyway so it's not stretch by any means to do that. b) Or just replicate changes to leaf partitions as changes to partitioned table. That's also not that hard to do, we just need fast lookup of partitioned table from leaf table. I guess a) looks simpler given that we eventually need the rename anyway, but I'd like opinions of other people as well. > I'm not sure but should adding/removing partitions after the fact cause > their addition/removal from the publication (& subscription)? Maybe we'll > discuss these issues later. That's something I've been also wondering as there is many corner cases here. For example if table is in some publication and then is added as partition to partitioned table what should happen? What should happen when the partitioned table is then removed from same publication to which partition was added explicitly? Should we allow different publication configuration for different partitions within same partitioned table, etc? This somewhat brings bigger question about where we want to go in general with partitioning. And that is, should partition be a separate entity that is on the same level of table, or should it be part of the partitioned table without it's own "identity". -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
pgsql-hackers by date: