Re: data consolidation: logical replication design considerations - Mailing list pgsql-performance

From Rory Campbell-Lange
Subject Re: data consolidation: logical replication design considerations
Date
Msg-id YtVs0blyucDTEfRF@campbell-lange.net
Whole thread Raw
In response to Re: data consolidation: logical replication design considerations  (Rick Otten <rottenwindfish@gmail.com>)
List pgsql-performance
On 17/07/22, Rick Otten (rottenwindfish@gmail.com) wrote:
> On Sat, Jul 16, 2022 at 12:07 PM Rory Campbell-Lange <
> rory@campbell-lange.net> wrote:
> 
> > I'd be grateful for some comments on the advisability of using a large
> > number of concurrent logical replication publications/subscriptions.
> > Below I've set out the current environment and a suggested design.
> > Apologies for the length of this email.
> 
> Another possibility is to use SymmetricDS for this.  [
> https://symmetricds.org ]  SymmetricDS was originally developed to keep
> databases on thousands of Point-of-Sale databases (in cash registers) in
> sync with pricing and inventory data for large international retailers.
> 
> There are lots of other use cases, but even 10-12 years ago it was scalable
> to the extent you are describing you need here.
> 
> The main drawback is that it is trigger based, so there is some slight
> latency introduced for insert/update/delete actions on the tables on the
> appropriate master, but it usually isn't significant.

Thanks very much for the pointer to SymmetricDS. I haven't come across
it before. The architecture, configuration and use look very
straightforward, although using java would be new to our production
environment, and SymmetricDS doesn't seem to be in Debian.

I'd be grateful to know if 500 odd publishers/subscribers is "out of the
park" or reasonable for a reasonably powerful machine (as described in
my original email). I would have thought using the native logical
replication capabilities would be much more scalable and efficient than
stepping outside of postgresql.

Regards,
Rory



pgsql-performance by date:

Previous
From: Rick Otten
Date:
Subject: Re: data consolidation: logical replication design considerations
Next
From: bruno da silva
Date:
Subject: PostgresSQL 9.5.21 very slow to connect and perform basic queries