Currently, the documentation explicitly states, that after failover, the old master must be recreated from scratch, or pg_rewind should be used (requiring wal_log_hints to be on, which is off by default):
> The former standby is now the primary, but the former primary is down and might stay down. To return to normal operation, a standby server must be recreated, either on the former primary system when it comes up, or on a third, possibly new, system. The pg_rewind utility can be used to speed up this process on large clusters.
My research shows that some people already rely on the following when planned failover (aka switchover) procedure, doing it in production:
1) shutdown the current master
2) ensure that the "master candidate" replica has received all WAL data including shutdown checkpoint from the old master
3) promote the master candidate to make it new master
4) configure recovery.conf on the old master node, while it's inactive
5) start the old master node as a new replica following the new master.
It looks to me now, that if no steps missed in the procedure, this approach is eligible for Postgres versions 9.3+ (for older versions like 9.3 maybe not really always – people who know details better will correct me here maybe). Am I right? Or I'm missing some risks here?
Two changes were made in 9.3 which allowed this approach in general [1] [2]. Also, I see from the code [3] that during shutdown process, the walsenders are the last who are stopped, so allow replicas to get the shutdown checkpoint information.
Is this approach considered as safe now?
if so, let's add it to the documentation, making it official. The patch is attached.
Links:
