Streaming replication failover with 3 servers - Mailing list pgsql-admin

From Stuart Bishop
Subject Streaming replication failover with 3 servers
Date
Msg-id CADmi=6N3qc6q=b=N=wxdGixkx9o-7+ccKS=DD1OXTThVzfk1YQ@mail.gmail.com
Whole thread Raw
Responses Re: Streaming replication failover with 3 servers
List pgsql-admin
Hi.

I'm trying to work out failover and disaster recovery procedures for a
cluster of three servers. Streaming replication is being used with a high
wal_keep_segments, no log shipping is happening. I need to avoid the
several hours it takes to rebuild a hot standby from scratch.

ServerA is the master.
ServerB is a streaming hot standby and prefered failover server.
ServerC is a streaming hot standby.


For a planned failover, maintenance on ServerA:

    1. Shutdown ServerB & ServerC
    2. Shutdown ServerA
    3. Copy pg_xlog from ServerA to ServerB and ServerC
    4. Reconfigure ServerB as master, start it up.
    5. Reconfigure ServerC as streaming hot standby of ServerB. Start it.
    6. After maintenance, reconfigure ServerA as streaming hot standby
       of ServerB. Start it.

For an unplanned failover, ServerA has exploded:

    1. Run 'SELECT pg_last_xlog_receive_location()' on ServerB and ServerC,
       determining which is most up to date.
    2. Shutdown ServerB and ServerC
    3. If ServerC is more up to date, copy pg_xlog from ServerC to ServerB.
    4. Reconfigure ServerB as master, start it up.
    5. Reconfigure ServerC as streaming hot standby of ServerB, start it up.


Does this look correct to people?

Am I going to end up in trouble copying files into pg_xlog like this on a
busy system?

Is it overengineered? eg. will a master ensure everything is streamed to
connected hot standbys before a graceful shutdown?

--
Stuart Bishop <stuart@stuartbishop.net>
http://www.stuartbishop.net/

pgsql-admin by date:

Previous
From: Craig Ringer
Date:
Subject: Re: Data migration to sql server 2008
Next
From: Amador Alvarez
Date:
Subject: Re: standby with a fixed lag behind the master