Postgres Pro
Downloads
Home   >   mailing lists

[GENERAL] pg_basebackup ----xlog-method=stream - Mailing list pgsql-general

From Ian Harding
Subject [GENERAL] pg_basebackup ----xlog-method=stream
Date
Msg-id CAMR4UwGruZ885xK_QThvBDxpPc+Xx=pFz1a7rDt0vrf+w_riGw@mail.gmail.com
Whole thread Raw
Responses Re: [GENERAL] pg_basebackup ----xlog-method=stream  (Adrian Klaver <adrian.klaver@aklaver.com>)
Re: [GENERAL] pg_basebackup ----xlog-method=stream  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-general
Tree view
I used this command to set up a streaming replica and it worked perfectly.

I tried to run it to create an online backup of the master on that replica for backup purposes and it seems not to have worked as well.

I thought that streaming the WAL would eliminate the need to keep tons of WAL around, that since they were streamed concurrently with the file they would be in place.  However, I got

pg_basebackup: could not get transaction log end position from server: ERROR:  requested WAL segment 000000010000496F000000BF has already been removed

The ...backup file in the archive logs folder of the master looks like this"

# cat 000000010000496F000000BF.00578DF8.backup
START WAL LOCATION: 496F/BF578DF8 (file 000000010000496F000000BF)
STOP WAL LOCATION: 4971/11FC6528 (file 000000010000497100000011)
CHECKPOINT LOCATION: 496F/C1ACE0D8
BACKUP METHOD: streamed
BACKUP FROM: master
START TIME: 2017-04-21 01:08:38 PDT
LABEL: full-20170421
STOP TIME: 2017-04-21 06:31:28 PDT

so the "...already been removed" message is referring to the very first file.

The files were definitely being streamed, I restarted and on the master I see this:

[root@db20 ~]# ps aux | grep stream
postgres 113855  0.4  0.0 274955352 2932 ?      Ss   10:53   0:00 postgres: wal sender process postgres 192.168.4.21(51292) streaming 4972/7FBC22B8
root     115958  0.0  0.0 112652   956 pts/2    S+   10:56   0:00 grep --color=auto stream
postgres 142072  0.3  0.0 274955348 3160 ?      Ss   Apr20   3:47 postgres: wal sender process postgres 192.168.4.21(51240) streaming 4972/7FBC22B8


although the --verbose output from pg_basebackup mentions starting the streaming, it doesn't mention the file names like it does for the datadir files.

Am I misunderstanding how this works?  I have WAL archiving set up, so the files are available, but I wanted them included in the backup.

Thanks!

- Ian

pgsql-general by date:

Previous
From: Edson Lidorio
Date:
Subject: Re: [GENERAL] Recover PostgreSQL database folder data
Next
From: Adrian Klaver
Date:
Subject: Re: [GENERAL] pg_basebackup ----xlog-method=stream