Re: Applying logical replication changes by more than one process - Mailing list pgsql-hackers

From konstantin knizhnik
Subject Re: Applying logical replication changes by more than one process
Date
Msg-id CEF1DD58-CAF0-4A56-B17F-CE65A71E02FB@postgrespro.ru
Whole thread Raw
In response to Re: Applying logical replication changes by more than one process  (Petr Jelinek <petr@2ndquadrant.com>)
List pgsql-hackers

On Mar 22, 2016, at 11:14 AM, Petr Jelinek wrote:

And each slot means connection with logical decoding attached to it so you don't really want to have thousands of those anyway. I think you'll hit other problems faster than loop over slots becomes problem if you plan to keep all of them active.


Assume that cluster have thousands of nodes and we use sharding to scatter data through cluster nodes.
But to provide HA we want to perform sharding with some level of redundancy, for example save the same record at 3 different nodes.
Once possible approach (pg_shard) is to execute the same query at three different shards.
But there is no warranty that  result of execution will be the same at all nodes.
Alternative approach is to execute transaction at one node and then replicate it using logical replication to replicas.
So we do not perform logical replication to all 1000 nodes. Just to 2 of them. But each time it will be different pair of nodes. So we still need to have 1000 active replication slots.

May be logical replication can not be used at all in such scenario - I have not thought much about it yet. Our first step will be multimaster without sharding.




--
 Petr Jelinek                  http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

pgsql-hackers by date:

Previous
From: konstantin knizhnik
Date:
Subject: Re: Applying logical replication changes by more than one process
Next
From: Mithun Cy
Date:
Subject: Re: POC: Cache data in GetSnapshotData()