Re: Timeline following for logical slots - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Timeline following for logical slots
Date
Msg-id CAMsr+YE=uyyDH-tcOeLATEBWStAK1gXnPpdq9F+8fTkK6OWRLw@mail.gmail.com
Whole thread Raw
In response to Re: Timeline following for logical slots  (Andres Freund <andres@anarazel.de>)
Responses Re: Timeline following for logical slots
List pgsql-hackers
On 1 April 2016 at 14:52, Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2016-04-01 08:46:01 +0200, Andres Freund wrote:
> That's a fundamental misunderstanding on your part (perhaps created by
> imprecise docs).

> > Speaking of which, did you see the proposed README I sent for
> > src/backend/replication/logical ?
>
> I skimmed it. But given we have a CF full of patches, some submitted
> over a year ago, it seems unfair to spend time on a patch submitted a
> few days ago.

For that purpos

WRT design readme, it might be interesting to look at 0009 in
http://archives.postgresql.org/message-id/20140127162006.GA25670%40awork2.anarazel.de

That's not up2date obviously, but it still might help.

Thanks, I've been reading it and the posts it references.

Most of it was familiar by this point, but would've been a good reference earlier on. The snapshot builder docs in README.SNAPBUILD.txt are handy and help glue a few separate pieces together better for me, and the invalidations section was brief but informative.

The very last point looks interesting, but only really alludes to what's going on:


+== Restartable Decoding ==
+
+As we want to generate a consistent stream of changes we need to have the
+ability to start from a previously decoded location without waiting possibly
+very long to reach consistency. For that reason we dump the current visibility
+information to disk whenever we read an xl_running_xacts record.

I don't feel like I've grasped this properly yet. I think it's referring to the pg_logical/snapshots/ serialization, the use of which allows us to avoid doing extra work in SnapBuildFindSnapshot(...), but doesn't seem to be crucial for correct function. After all, decoding still restarts at the restart_lsn and feeds relevant xact info into the snapshot builder, accumulates invalidation information, etc.


After 9.6 I'd like to go through that, update it, and get it in as a README for logical decoding. It would've done me a lot of good when getting up to speed.

 



--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby
Next
From: Andres Freund
Date:
Subject: Re: Timeline following for logical slots