Re: WIP: Failover Slots - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: WIP: Failover Slots
Date
Msg-id CAMsr+YHYwGyk4j3C17PZAMzcH6J01AvnZ2EMj6D+MvP_P87Uzg@mail.gmail.com
Whole thread Raw
In response to Re: WIP: Failover Slots  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: WIP: Failover Slots  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On 22 January 2016 at 00:31, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jan 1, 2016 at 7:50 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> Failover Slots
> In the current patch, any slot defined on a master will generate WAL,
> leading to a pending-slot being present on all standby nodes. When a standby
> is promoted, the slot becomes usable and will have the properties as of the
> last fsync on the master.

No objection to the concept, but I think the new behavior needs to be
optional.  I am guessing that you are thinking along similar lines,
but it's not explicitly called out here.

Yeah,  I think that's the idea too. For one thing we'll want to allow non-failover slots to continue to be usable from a streaming replica, but we must ERROR if anyone attempts to attach to and replay from a failover slot via a replica since we can't write WAL there. Both kinds are needed.

There's a 'failover' bool member in the slot persistent data for that reason. It's not (yet) exposed via the UI.

I presume we'll want to:

* add a new default-false argument is_failover_slot or similar to pg_create_logical_replication_slot and pg_create_physical_replication_slot

* Add a new optional flag argument FAILOVER to CREATE_REPLICATION_SLOT in both its LOGICAL and PHYSICAL forms.

... and will be adding that to this patch, barring syntax objections etc.


It's also going to be necessary to handle what happens when a new failover slot (physical or logical) is created on the master where it conflicts with the name of a non-failover physical slot that was created on the replica. In this case I am inclined to terminate any walsender backend for the replica's slot with a conflict with recovery, remove its slot and replace it with a failover slot. The failover slot does not permit replay while in recovery so if the booted-off client reconnects it'll get an ERROR saying it can't replay from a failover slot. It should be pretty clear to the admin what's happening between the conflict with recovery and the failover slot error. There could still be an issue if the client persistently keeps retrying and successfully reconnects after replica promotion but I don't see that much can be done about that. The documentation will need to address the need to try to avoid name conflicts between slots created on replicas and failover slots on the master.

I'll be working on those docs, on the name conflict handling, and on the syntax during my coming flight. Toddler permitting.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Pavel Stehule
Date:
Subject: Re: custom function for converting human readable sizes to bytes
Next
From: Tomas Vondra
Date:
Subject: Re: silent data loss with ext4 / all current versions