Re: Timeline following for logical slots - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Timeline following for logical slots
Date
Msg-id CAMsr+YGmcO8fHif4docwdRcC8HCvXMcuMK6kYS4e2U3Tuvg2eg@mail.gmail.com
Whole thread Raw
In response to Re: Timeline following for logical slots  (Andres Freund <andres@anarazel.de>)
Responses Re: Timeline following for logical slots  (Craig Ringer <craig@2ndquadrant.com>)
List pgsql-hackers
On 28 April 2016 at 02:29, Andres Freund <andres@anarazel.de> wrote:
 

I don't object either, I was looking for the feature myself a number of
times (for tap tests in my case).

Exactly what I want it for.
 
It requires a some amount of thinking about what the limit applies to
though. "messages sent by server", Bytes? TCP messages? xids? Time?


The main thing I think would be useful is a threshold for message LSNs after which it disconnects and exits. It doesn't have to be a commit message; if we receive any message with upstream LSN after the LSN of interest then there can't be any commits with a later LSN anyway, and that way it'll be useful if/when streaming decoding is implemented too.

That way a test can capture the xlog insert lsn after doing the work it wants to replay, do another dummy write to make sure there's something more to replay, and decode up to that point.

I don't think pg_recvlogical can do anything about the need for that dummy write, since the client has no way to determine the exact LSN of the commit record of the xact of interest. It can't rely on pg_current_xlog_insert_location() or pg_current_xlog_location() since autovacuum or a checkpoint might've written xlog since. Logical streaming replication doesn't have a non-blocking mode where it returns immediately if it'd have to wait for more xlog so we can't just send off the most recent server LSN as the endpoint.

Ideally I think this should be done server-side with a limit on replay LSN and a non-blocking option, but that's way too invasive for this stage in the release cycle. Client-side seems fine enough.

Number of xacts received would also be handy, but I don't know if I'll get to that.

pg_receivexlog should send confirmation on exit.

(The other thing I've wanted sometimes is a --peek option, but that's again not really in scope for this.)


--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: [sqlsmith] Failed assertion in BecomeLockGroupLeader
Next
From: Stefan Huehner
Date:
Subject: pg9.6 segfault using simple query (related to use fk for join estimates)