Re: Logical decoding on standby - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Logical decoding on standby
Date
Msg-id CAMsr+YGFBm_i9R341E_ANkkyHOm6NbpNZZR8kw5Xv+JPx_69cw@mail.gmail.com
Whole thread Raw
In response to Logical decoding on standby  (Дмитрий Сарафанников<dimon99901@mail.ru>)
Responses Re: Logical decoding on standby  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 19 January 2016 at 23:30, Дмитрий Сарафанников <dimon99901@mail.ru> wrote:

When you plan to add logical decoding on standby?

 
it is useful to have separate standby server for logical replication that will not break the master if you make a mistake in plugin.

Indeed.  It might PANIC it and force a restart, if that's what you mean. I guess in an extreme case you could clobber shmem silently, which would be bad. (I wonder if we could make shared_buffers mmap'ed read-only for walsender backends?). So it certainly seems like it'd be nice to have. There's also the advantage that you could shift load off the master by having physical master=>replica replication on a fast wide link, then do logical decoding from there to send over the wire to WAN sites etc.

Unfortunately it's not particularly simple and nobody seems to have time to implement it. As Álvaro pointed out, sometimes you have to do the work if you want the change to happen. Or find someone with the existing skills and convince them to want to do it, but most of those people are already very, very busy.

As part of the failover slots work Simon noted that:

"To prevent needed rows from being removed we need we would need to enhance hot_standby_feedback so it sends both xmin and catalog_xmin to the master."

... which means a protocol change in the walsender protocol. So you're looking at that plus the other comment given above, that

"We need to be able to correctly and quickly identify the timeline a  LSN belongs to"

.... which is new internal infrastructure and new state in the replica that must be maintained. Probably persistently.

In other words ... this isn't especially simple to do. It requires at least two non-trivial core changes. Nobody who has the skills to do it reasonably quickly also has the time to do it at all or has other higher priority things to do first. So it's not likely to happen soon unless you or someone like you steps up and has a go at it.

If you do decide to attempt to implement it and you're willing to read a fair bit of code and documentation to do so you'll find people here pretty helpful with suggestions, ideas, and help if you get stuck. But only if you put in the work yourself.

(On a side note, failover slots probably won't be usable from a standby either. They have to write to WAL and you can't write to WAL from a standby. So if slot support is ever added on a standby it'll probably be ephemeral slots only.)

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Craig Ringer
Date:
Subject: Re: Stream consistent snapshot via a logical decoding plugin as a series of INSERTs
Next
From: Craig Ringer
Date:
Subject: Re: Inconsistent error handling in START_REPLICATION command