Thread: Bringing up new slaves faster

Bringing up new slaves faster

From
François Deliège
Date:
Hi all,

Bringing up new slaves is a task we are performing very frequently.  Our current process is: 
1. launch a new instance, perform a pg_basebackup
2. start postgresql on the slave and let the slave recover from the archive all the xlog files it missed and that got created during step 1.
3. let the slave stream from the master

We use pg_basebackup with the -X option to stream the wal files created and get rid of step 2.  

However, when the postgresql server starts on the slave, it first tries to recover the files from our archive even if the files have been copied to the pg_xlog directory.   Recovering from the archive is slow and seem to be unnecessary given that the xlogs have been copied.  

To address that problem we wrote a little shell script that changes that order.  The script let the slave make a first attempt at using the local xlog file if it exists, and falls back to recovering from the archive in case of failure.

While this seems to be a common use case, we could not find much info about this.  
- Are we missing something? 
- What are the trade-offs of such approach?  
- Is there any additional risks of bringing up new slave with that recovery option?

Thanks a lot!

Francois