Re: Design for In-Core Logical Replication - Mailing list pgsql-hackers

From Rod Taylor
Subject Re: Design for In-Core Logical Replication
Date
Msg-id CAKddOFCdzL=qVNUS628kgeVdWKJd9Udf4bRpit825rZeJgNVXQ@mail.gmail.com
Whole thread Raw
In response to Design for In-Core Logical Replication  (Simon Riggs <simon@2ndquadrant.com>)
Responses Re: Design for In-Core Logical Replication  (Simon Riggs <simon@2ndquadrant.com>)
List pgsql-hackers


On Wed, Jul 20, 2016 at 4:08 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
 

  <para>
    And on Subscriber database:
<programlisting>
CREATE SUBSCRIPTION mysub WITH CONNECTION <quote>dbname=foo host=bar user=repuser</quote> PUBLICATION mypub;
</programlisting>
  </para>
  <para>
    The above will start the replication process which synchronizes the
    initial table contents of <literal>users</literal> and
    <literal>departments</literal> tables and then starts replicating
    incremental changes to those tables.
  </para>
</sect1>
</chapter>

I think it's important for communication channels to be defined separately from the subscriptions.

If I have nodes 1/2 + 3/4 which operate in pairs, I don't really want to have to have a script reconfigure replication on 3/4 every-time we do maintenance on 1 or 2.

3/4 need to know they subscribe to mypub and that they have connections to machine 1 and machine 2. The replication system should be able to figure out which (of 1/2) has the most recently available data.


So, I'd rather have:

CREATE CONNECTION machine1;
CREATE CONNECTION machine2;
CREATE SUBSCRIPTION TO PUBLICATION mypub;

Notice I explicitly did not tell it how to get the publication but if we did have a preference the DNS weighting model might be appropriate.

I'm not certain the subscription needs to be named. IMO, a publication should have the same properties on all nodes (so any node may become the primary source). If a subscriber needs different behaviour for a publication, it should be created as a different publication.

Documenting that ThisPub is different from ThatPub is easier than documenting that ThisPub on node 1/2/4 is different from ThisPub on node 7/8, except Node 7 is temporarily on Node 4 too (database X instead of database Y) due to that power problem.


Clearly this is advanced. An initial implementation may only allow mypub from a single connection.


I also suspect multiple publications will be normal even if only 2 nodes. Old slow moving data almost always got different treatment than fast-moving data; even if only defining which set needs to hit the other node first and which set can trickle through later.

regards,

Rod Taylor

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: skink's test_decoding failures in 9.4 branch
Next
From: Teodor Sigaev
Date:
Subject: Re: One process per session lack of sharing