Thread: Missing WALs when doing pg_basebackup from slave...

I have been testing a master slave configuration of PostgreSQL 9.4.2 using
streaming replciation. The master is initialised with a blank database and
the slave is created with a pg_start_backup()/rsync/pg_stop_backup()
procedure. The slave connects to the master and works flawlessly. For
testing purposes backup is configured to run every 5 minutes from the
master on one location, and from the slave on an other location. The backup
procedure is done using pg_basebackup with the --xlog-method=stream to
fetch all WALs that are needed for the backup to be fully functional.

I left it running for a day. A few backups that were made from the slave
didn't contain any WAL files (only history files in complex cases when the
master had switched to a newer timeline).

Is it normal that pg_basebackup runs successfully (rc=0) and there is no
WAL files present?

The master and slave are sitting idle, after only a few transaction on the
master at the beginning of the day. I noted that all WAL switches are
caused by the backup running on the master. Is it possible the slave is in
a consistent state when it has applied all changes from the previous WAL
and the new WAL hasn't been created yet on the master (so actually no WAL-s
are needed to restore it to a consistent state)?

Regards,
Mladen Marinović

On Thu, 11 Jun 2015 14:48:44 +1000, Venkata Balaji N <nag1010@gmail.com>
wrote:
> On Wed, Jun 10, 2015 at 6:47 PM,  wrote:
>  Is it normal that pg_basebackup runs successfully (rc=0) and there is
no
>  WAL files present?
>
> Yes, it is normal. "pg_basebackup" ensures that required WALs are backed
> along with the data directory. This is to ensure backup is consistent.

The backup pg_basebackup made had NO WAL files. I tried to restored but it
failed without at lest the WAL file mentioned in the backup_label file. It
is my understanding that pg_basebackup --xlog-method=stream should stream
the needed WAL files, and it usually does. But as I mentioned, I had it run
all day every 5 minutes and 3/100 backups were without WAL files. I am not
sure if this is a glitch of pg_basebackup or what, but I expected at least
an error result code (rc!=0).

>   The master and slave are sitting idle, after only a few transaction on
> the
>  master at the beginning of the day. I noted that all WAL switches are
>  caused by the backup running on the master. Is it possible the slave is
> in
>  a consistent state when it has applied all changes from the previous
WAL
>  and the new WAL hasn't been created yet on the master (so actually no
> WAL-s
>  are needed to restore it to a consistent state)?
>  I am not sure if I got your question correct. The amount of
> transactions in the master database may be low or high, the WALs will be
> replicated to slave.To ensure slave is receiving all the WALs, you need
> to check the sync status between master and slave. If there is no new
WAL
> generated at master, then slave must be in consistent state should have
> applied all the previous WALs, if not, then all the previous WALs are
> needed to get the slave to a consistent state. Nothing can be advised
> straight without knowing your replication configuration/architecture
> details.
> Regards,Venkata Balaji N
> Fujitsu Australia
>
>
>
> Links:
> ------
> [1] mailto:marin@kset.org