Thread: Wal standbies switchover
Goal - having 1 master and 2 standbies - make switchover in the way , that except minimal downtime - there was always 1 running master and 2 warm standbies.
Lets say i have 3 hosts (1,2,3) , 1 master and 2 warm standbies . ( M1 SB2 SB3)
Each SB gets (Master rsyncs from his side to SB) archived wal files from master and recovers in continues way untill triggered to start as master., also it gets incomplete wals from pg_xlog dir .
Switchover looks for me like this.
1. M1 stops , all sync crons stopped.
2. Do last rsync of archived wals and pg_xlog to SB2 , SB3
3. SB2 triggered to start , it recovers last incomplete wals from pg_xlog and starts (becomes M2), and starts cron rsync to SB3
4. SB3 does nothing , it continues get recover wal archives which now it gets from M2. (Question - is it ok ? It looks like new archived wal files from M2 should be valid without any changes)
5. What happens on host 1 (former master) is a question. It needs to become a SB1 . Do i need to make large rsync of whole db from M2 (like during SB install) or i can just keep old data and start db in recovery mode , getting new wal files?
There is a difference between SB2 and new SB1. SB2 had data_dir - corresponding to last recovered ARCHIVED file , and SB1 - to last INCOMPLETE WAL files , so its not the same.
So , 2 questions on points 4 and 5.
Any comments?
Lets say i have 3 hosts (1,2,3) , 1 master and 2 warm standbies . ( M1 SB2 SB3)
Each SB gets (Master rsyncs from his side to SB) archived wal files from master and recovers in continues way untill triggered to start as master., also it gets incomplete wals from pg_xlog dir .
Switchover looks for me like this.
1. M1 stops , all sync crons stopped.
2. Do last rsync of archived wals and pg_xlog to SB2 , SB3
3. SB2 triggered to start , it recovers last incomplete wals from pg_xlog and starts (becomes M2), and starts cron rsync to SB3
4. SB3 does nothing , it continues get recover wal archives which now it gets from M2. (Question - is it ok ? It looks like new archived wal files from M2 should be valid without any changes)
5. What happens on host 1 (former master) is a question. It needs to become a SB1 . Do i need to make large rsync of whole db from M2 (like during SB install) or i can just keep old data and start db in recovery mode , getting new wal files?
There is a difference between SB2 and new SB1. SB2 had data_dir - corresponding to last recovered ARCHIVED file , and SB1 - to last INCOMPLETE WAL files , so its not the same.
So , 2 questions on points 4 and 5.
Any comments?
On Thu, Aug 02, 2007 at 12:26:47PM +0400, Dimitri Gritsenko wrote: > Goal - having 1 master and 2 standbies - make switchover in the way , that > except minimal downtime - there was always 1 running master and 2 warm > standbies. > > Lets say i have 3 hosts (1,2,3) , 1 master and 2 warm standbies . ( M1 SB2 > SB3) > Each SB gets (Master rsyncs from his side to SB) archived wal files from > master and recovers in continues way untill triggered to start as master., > also it gets incomplete wals from pg_xlog dir . > Switchover looks for me like this. > 1. M1 stops , all sync crons stopped. > 2. Do last rsync of archived wals and pg_xlog to SB2 , SB3 > 3. SB2 triggered to start , it recovers last incomplete wals from pg_xlog > and starts (becomes M2), and starts cron rsync to SB3 > 4. SB3 does nothing , it continues get recover wal archives which now it > gets from M2. (Question - is it ok ? It looks like new archived wal files > from M2 should be valid without any changes) Hrm... I'm not sure if that's OK... I seem to recall something about a difference in WAL file numbering when a PITR recovery is done (promoting slave to master in your case). I suggest trying it; if it works I don't expect there to be any hidden issues. > 5. What happens on host 1 (former master) is a question. It needs to become > a SB1 . Do i need to make large rsync of whole db from M2 (like during SB > install) or i can just keep old data and start db in recovery mode , getting > new wal files? No, you can't take a data directory that's been run outside of recovery mode and push it back into a recovery mode (which is essentially what you're asking for). You'll need to rsync from host 2 to host 1 and start recovery from there. There's a very critical thing you've overlooked, too. You have to ensure that slaves will *only* get archived logs from the master. This means that if you switch masters, you have to guarantee that the old master won't be archiving any logfiles anymore. Another possibility would be to ensure that things are configured so that each machine will archive to a different location. -- Decibel!, aka Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)