Thread: WAL Info messages
One of the reasons for last years Rmgr plugin patch was the idea of a signalling interface between Primary and Standby nodes. Such a feature would help in producing further programmatic tests for Hot Standby. Proposal is to use up the last rmgr id, slot 7: RM_INFO_ID. Functions would be provided on primary side to send a user-defined binary message into WAL stream, with a single message type XLOG_INFO_MSG. (Superuser only, though can be wrapped and given to trusted users). On Standby side, we would provide a plugin interface to allow the message to be read and handled by user-written code. This would then allow a simple value or notification to be passed to a waiting (polling) connection on standby. If new LISTEN/NOTIFY gets into 8.5 then this would be an obvious fit, but I have no need for message passing, just info passing, so no need to wait for that; something much simpler. We could do this by adding stuff into Streaming Rep patch, but seems best to provide a generic interface that lives in the WAL stream itself, rather than being at a meta level and divorced from the WAL. This would allow the writing of a test suite that can create certain conditions on the primary, then send notification of the event to the standby. Standby would then halt recovery in a precise state while a user runs an appropriate test. -- Simon Riggs www.2ndQuadrant.com
On Sun, Dec 13, 2009 at 2:20 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > This would allow the writing of a test suite that can create certain > conditions on the primary, then send notification of the event to the > standby. Standby would then halt recovery in a precise state while a > user runs an appropriate test. That sounds neat. What kinds of conditions did you have in mind? ...Robert
Simon Riggs <simon@2ndQuadrant.com> writes:
> Proposal is to use up the last rmgr id, slot 7: RM_INFO_ID.
> Functions would be provided on primary side to send a user-defined
> binary message into WAL stream, with a single message type
> XLOG_INFO_MSG. (Superuser only, though can be wrapped and given to
> trusted users).
Seems like a frammish that we could maybe worry about a few months from
now, after the core patch has settled down.
I also dislike reserving a whole RM_ID just for this. If we are only
going to send one type of message it seems like it could be one infomask
type within an existing RM_ID.
regards, tom lane
On Sun, Dec 13, 2009 at 7:20 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On Standby side, we would provide a plugin interface to allow the > message to be read and handled by user-written code. This would then > allow a simple value or notification to be passed to a waiting (polling) > connection on standby. If new LISTEN/NOTIFY gets into 8.5 then this > would be an obvious fit, but I have no need for message passing, just > info passing, so no need to wait for that; something much simpler. What happens on the slave when normal NOTIFYs are generated on the master? IIRC NOTIFYs are wal-logged so I imagine LISTEN on the slave would just work and a plain old NOTIFY/LISTEN would suffice for this use case? -- greg
Simon Riggs wrote: > Proposal is to use up the last rmgr id, slot 7: RM_INFO_ID. Rmgr id is a 8-bit integer, there's plenty of free rmgr ids left. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Greg Stark <gsstark@mit.edu> writes:
> What happens on the slave when normal NOTIFYs are generated on the
> master? IIRC NOTIFYs are wal-logged so I imagine LISTEN on the slave
> would just work and a plain old NOTIFY/LISTEN would suffice for this
> use case?
Nothing happens: NOTIFYs are *not* WAL-logged, and it would be quite
against the intended use-case to have them be.
If you're hoping that processes on the slave could LISTEN for events
happening on the master, we'd have a bit of a conflict. This might be
an example of a case where we need that "are we doing HS" flag that
Simon insists we don't need.
regards, tom lane
On Mon, 2009-12-14 at 09:51 -0500, Tom Lane wrote: > This might be > an example of a case where we need that "are we doing HS" flag that > Simon insists we don't need. Simon has put it there at your request, in the place you suggested. Others did speak against it, not me. -- Simon Riggs www.2ndQuadrant.com
On Mon, 2009-12-14 at 09:51 -0500, Tom Lane wrote: > Greg Stark <gsstark@mit.edu> writes: > > What happens on the slave when normal NOTIFYs are generated on the > > master? IIRC NOTIFYs are wal-logged so I imagine LISTEN on the slave > > would just work and a plain old NOTIFY/LISTEN would suffice for this > > use case? > > Nothing happens: NOTIFYs are *not* WAL-logged, and it would be quite > against the intended use-case to have them be. > > If you're hoping that processes on the slave could LISTEN for events > happening on the master, we'd have a bit of a conflict. What I've proposed is essentially a variant of NOTIFY on master, LISTEN on slave, as Greg suggests. Almost identical if we have NOTIFY-with-payload. I definitely wouldn't presume that anybody using Hot Standby would necessarily want NOTIFY to reach the standby, especially if there was an overhead to doing so. If using NOTIFY is the favoured approach, I would add a separate parameter for it and/or an explicit option on NOTIFY. -- Simon Riggs www.2ndQuadrant.com
Simon Riggs <simon@2ndQuadrant.com> writes:
> I definitely wouldn't presume that anybody using Hot Standby would
> necessarily want NOTIFY to reach the standby, especially if there was an
> overhead to doing so. If using NOTIFY is the favoured approach, I would
> add a separate parameter for it and/or an explicit option on NOTIFY.
Yeah, I had just come to the same conclusion: you'd want a separate
spigot handle on sending NOTIFY to WAL, regardless of any general flag
about HS.
regards, tom lane
On Mon, 2009-12-14 at 11:06 -0500, Tom Lane wrote: > Simon Riggs <simon@2ndQuadrant.com> writes: > > I definitely wouldn't presume that anybody using Hot Standby would > > necessarily want NOTIFY to reach the standby, especially if there was an > > overhead to doing so. If using NOTIFY is the favoured approach, I would > > add a separate parameter for it and/or an explicit option on NOTIFY. > > Yeah, I had just come to the same conclusion: you'd want a separate > spigot handle on sending NOTIFY to WAL, regardless of any general flag > about HS. Anybody know the latest on in-memory NOTIFY? -- Simon Riggs www.2ndQuadrant.com
On tis, 2009-12-15 at 03:34 +0000, Simon Riggs wrote: > Anybody know the latest on in-memory NOTIFY? "Returned with Feedback"