Bringing up new slaves faster - Mailing list pgsql-performance

From François Deliège
Subject Bringing up new slaves faster
Date
Msg-id CAPMnNqdwheZZ=d939MueS-i50XUUpO9YbH=qU-LrE=SeQ02R6Q@mail.gmail.com
Whole thread Raw
List pgsql-performance
Hi all,

Bringing up new slaves is a task we are performing very frequently.  Our current process is: 
1. launch a new instance, perform a pg_basebackup
2. start postgresql on the slave and let the slave recover from the archive all the xlog files it missed and that got created during step 1.
3. let the slave stream from the master

We use pg_basebackup with the -X option to stream the wal files created and get rid of step 2.  

However, when the postgresql server starts on the slave, it first tries to recover the files from our archive even if the files have been copied to the pg_xlog directory.   Recovering from the archive is slow and seem to be unnecessary given that the xlogs have been copied.  

To address that problem we wrote a little shell script that changes that order.  The script let the slave make a first attempt at using the local xlog file if it exists, and falls back to recovering from the archive in case of failure.

While this seems to be a common use case, we could not find much info about this.  
- Are we missing something? 
- What are the trade-offs of such approach?  
- Is there any additional risks of bringing up new slave with that recovery option?

Thanks a lot!

Francois

pgsql-performance by date:

Previous
From: didier
Date:
Subject: Re: Planner performance extremely affected by an hanging transaction (20-30 times)?
Next
From: Merlin Moncure
Date:
Subject: Re: Slow plan for MAX/MIN or LIMIT 1?