Re: Logical WAL streaming & START_REPLICATION - Mailing list pgsql-jdbc

From Craig Ringer
Subject Re: Logical WAL streaming & START_REPLICATION
Date
Msg-id CAMsr+YG76+soR1tpJafAC8ix0+OiFrJLxPo9L3U2vOzpE=+STA@mail.gmail.com
Whole thread Raw
In response to Logical WAL streaming & START_REPLICATION  (Joe Hammerman <jhammerman@squarespace.com>)
Responses Re: Logical WAL streaming & START_REPLICATION  (Craig Ringer <craig@2ndquadrant.com>)
List pgsql-jdbc
On 9 March 2018 at 04:49, Joe Hammerman <jhammerman@squarespace.com> wrote:
 
Although START_REPLICATION is called with the x_log position we desire, streaming is picked up from the current WAL log position.

That's how replication slots work. It's necessary to allow proper client- and server-side crash recovery. They're really intended for use with a replication origin or something similar client side to track client progress.
 
Is there a method to achieve our ends? Our alternative plan is to add a republish_count field to the the transaction, so that we can craft an SQL statement to update the rows from say, an hour ago to the present time.

What you'll need to do is find a way to delay sending replication slot replay confirmations  until you know the change is recorded persistently in Kafka.

>  that a flushed LSN may have been in the Kafka Broker Pool but not consumed. 

If your system will forget work on crash, it's not flushed, and you shouldn't report it flushed.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-jdbc by date:

Previous
From: Joe Hammerman
Date:
Subject: Logical WAL streaming & START_REPLICATION
Next
From: Craig Ringer
Date:
Subject: Re: Logical WAL streaming & START_REPLICATION