Re: [HACKERS] Time based lag tracking for logical replication - Mailing list pgsql-hackers

From Noah Misch
Subject Re: [HACKERS] Time based lag tracking for logical replication
Date
Msg-id 20170505070726.GA843225@rfd.leadboat.com
Whole thread Raw
In response to Re: [HACKERS] Time based lag tracking for logical replication  (Simon Riggs <simon@2ndquadrant.com>)
Responses Re: [HACKERS] Time based lag tracking for logical replication  (Noah Misch <noah@leadboat.com>)
List pgsql-hackers
On Wed, May 03, 2017 at 08:28:53AM +0200, Simon Riggs wrote:
> On 23 April 2017 at 01:10, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
> > Hi,
> >
> > The time based lag tracking commit [1] added interface for logging
> > progress of replication so that we can report lag as time interval
> > instead of just bytes. But the patch didn't contain patch for the
> > builtin logical replication.
> >
> > So I wrote something that implements this. I didn't like all that much
> > the API layering in terms of exporting the walsender's LagTrackerWrite()
> > for use by plugin directly. Normally output plugin does not have to care
> > if it's running under walsender or not, it uses abstracted write
> > interface for that which can be implemented in various ways (that's how
> > we implement SQL interface to logical decoding after all). So I decided
> > to add another function to the logical decoding write api called
> > update_progress and call that one from the output plugin. The walsender
> > then implements that new API to call the LagTrackerWrite() while the SQL
> > interface just does not implement it at all. This seems like cleaner way
> > of doing it.
> >
> > Thoughts?
> 
> Agree cleaner.
> 
> I don't see any pacing or comments about it, nor handling of
> intermediate messages while we process a large transaction.
> 
> I'll look at adding some pacing code in WalSndUpdateProgress()

[Action required within three days.]

Simon, I understand, from an annotation on the open items list, that you have
volunteered to own this item.  Please observe the policy on open item
ownership[1] and send a status update within three calendar days of this
message.  Include a date for your subsequent status update.  Testers may
discover new open items at any time, and I want to plan to get them all fixed
well in advance of shipping v10.  Consequently, I will appreciate your efforts
toward speedy resolution.  Thanks.

[1] https://www.postgresql.org/message-id/20170404140717.GA2675809%40tornado.leadboat.com



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkipcheckpoints, archiving on idle systems.)
Next
From: Noah Misch
Date:
Subject: Re: [HACKERS] Logical replication ApplyContext bloat