Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" - Mailing list pgsql-admin

On Mon, Dec 5, 2016 at 3:27 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> wrote:

Hi John,

 

Thank you for the feed back. Presently it is taking around 1hr to finish the backups. We have the wal_keep_segments set to 50. Is there way to adjust the segment size to be more than 16MB? At times I notice in the pg_xlog, there are 10-15 segments on an invocation point.

 

 

Thanx,

Sam

 

From: John Scalia [mailto:jayknowsunix@gmail.com]
Sent: Monday, December 05, 2016 1:52 PM
To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"

 

More than likely your keep_wal_segments setting is too low. How long is the pg_basebackup taking? Remember that segments are generated about every 5 minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.

--

Jay

Sent from my iPad

On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> wrote:

Hi

 

I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6.  When we do the postgres backups we are noticing the following errors

 

pg_basebackup: could not get WAL end position from server: FATAL:  requested WAL segment  000000010000020B00000040 has already been removed

 

But when I look into pg_xlog, I notice

 

000000010000020B00000040.00000118.backup

 

 

May I know how to resolve this error? Are the backups done with these errors, recoverable?

 

 

Thanks in advance for your support.

 

Thanx,

Sam

 

 

 

 

If you're using pgbasebackup just for creating consistent backups, use the --xlog-method=strean (-Xs) to ensure you get all the WAL files necessary for a consistent backup. This avoids having to set wal_keep_segments high enough to account for the time length of your entire backup. Note you will need an extra WAL stream for this setting, so ensure max_wal_senders is set high enough on the master.

Now if you're going to be using this backup to create a slave, you must get that slave up and running before the master recycles all the WAL files necessary for replication to be able to catch up from when the backup stopped. Otherwise you will see that error you shared in your original post. That's when you may need to adjust wal_keep_segments. Either that or you should set up an archive_command on the master to keep a secondary copy of all WAL files.

Keith