Re: Timeline following for logical slots - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Timeline following for logical slots
Date
Msg-id CAMsr+YG4B2R7Bfa7UgeVSb1fB2aACzTrOd_kJd9WEnuPna5abA@mail.gmail.com
Whole thread Raw
In response to Re: Timeline following for logical slots  (Craig Ringer <craig@2ndquadrant.com>)
Responses Re: Timeline following for logical slots  (Noah Misch <noah@leadboat.com>)
Re: Timeline following for logical slots  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Re: Timeline following for logical slots  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-hackers
Hi all

I made an unfortunate oversight in the logical decoding timeline following code: it doesn't work for logical decoding from the walsender, because I left the glue code required out of the final cut of the patch.

The reason the new src/test/recovery/ tests don't notice this is that using pg_recvlogical from the TAP tests is currently pretty awkward. pg_recvlogical has no way to specify a stop-point or return when there's no immediately pending data like the SQL interface does. So you have to read from it until you figure it's not going to return anything more then kill it and look at what it did return and hope you don't lose anything in buffering.I don't much like relying on timeouts as part of normal successful results since they can upset some of the older and slower buildfarm members. I'd rather be able to pass a --stoppos= or a --n-xacts= option, but it was a bit too late to add those.

Per the separate mail I sent to hackers, xlogreader is currently pretty much timeline-agnostic. The timeline tracking code needs to know when the xlogreader does a random read and do a fresh lookup so its state is part of the XLogReaderState struct. But the walsender's xlogreader page read callback doesn't use the xlogreader's state, and it can't because it's also used for physical replication, which communicates the timeline to read from with the page read function via globals. So for now, I just set the globals before calling the page read function:

+       XLogReadDetermineTimeline(state);
+       sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID;
+       sendTimeLine = state->currTLI;
+       sendTimeLineValidUpto = state->currTLIValidUntil;
+       sendTimeLineNextTLI = state->nextTLI;

Per separate mail, I'd quite like to tackle the level of duplication in timeline following logic in 9.7 and maybe see if the _three_ separate read xlog page functions can be unified at the same time. But that sure isn't 9.6 material.

Attachment

pgsql-hackers by date:

Previous
From: Craig Ringer
Date:
Subject: Timeline following is a bit tangled...
Next
From: Tom Lane
Date:
Subject: Re: EXPLAIN VERBOSE with parallel Aggregate