Re: Synchronous replay take III - Mailing list pgsql-hackers

From Michail Nikolaev
Subject Re: Synchronous replay take III
Date
Msg-id CANtu0oh2x+618NVZtGqLoehA0yJATKkNOBShCAc=HdXCCttkXg@mail.gmail.com
Whole thread Raw
In response to Re: Synchronous replay take III  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Synchronous replay take III  (Thomas Munro <thomas.munro@enterprisedb.com>)
List pgsql-hackers
Hello.

It is really nice feature. I am working on the project which heavily reads from replicas (6 of them).

In our case we have implemented some kind of "replication barrier" functionality based on table with counters (one counter per application backend in simple case).
Each application backend have dedicated connection to each replica. And it selects its counter value few times (2-100) per second from each replica in background process (depending on how often replication barrier is used).

Once application have committed transaction it may want join replication barrier before return new data to a user. So, it increments counter in the table and waits until all replicas have replayed that value according to background monitoring process. Of course timeout, replicas health checks and few optimizations and circuit breakers are used.

Nice thing here - constant number of connection involved. Even if lot of threads joining replication barrier in the moment. Even if some replicas are lagging.

Because 2-5 seconds lag of some replica will lead to out of connections issue in few milliseconds in case of implementation described in this thread.
It may be the weak part of the patch I think. At least for our case. But it possible could be used to eliminate odd table with counters in my case (if it possible to change setting per transaction).

Thanks a lot,
Michail.

pgsql-hackers by date:

Previous
From: Sergei Agalakov
Date:
Subject: Re: {PROPOSAL] add session information column to pg_stat_statements
Next
From: Thomas Munro
Date:
Subject: Re: dsa_allocate() faliure