pgsql: Make pg_receivexlog and pg_basebackup -X stream work across time - Mailing list pgsql-committers

From Heikki Linnakangas
Subject pgsql: Make pg_receivexlog and pg_basebackup -X stream work across time
Date
Msg-id E1Tvu9T-00076h-HP@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Make pg_receivexlog and pg_basebackup -X stream work across timeline switches.

This mirrors the changes done earlier to the server in standby mode. When
receivelog reaches the end of a timeline, as reported by the server, it
fetches the timeline history file of the next timeline, and restarts
streaming from the new timeline by issuing a new START_STREAMING command.

When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the
last segment on the old timeline. This helps you to tell apart a partial
segment left in the directory because of a timeline switch, and a completed
segment. If you just follow a single server, it won't make a difference, but
it can be significant in more complicated scenarios where new WAL is still
generated on the old timeline.

This includes two small changes to the streaming replication protocol:
First, when you reach the end of timeline while streaming, the server now
sends the TLI of the next timeline in the server's history to the client.
pg_receivexlog uses that as the next timeline, so that it doesn't need to
parse the timeline history file like a standby server does. Second, when
BASE_BACKUP command sends the begin and end WAL positions, it now also sends
the timeline IDs corresponding the positions.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/0b6329130e8e4576e97ff763f0e773347e1a88af

Modified Files
--------------
doc/src/sgml/protocol.sgml             |   14 +-
src/backend/access/transam/timeline.c  |   14 +-
src/backend/access/transam/xlog.c      |   24 +-
src/backend/access/transam/xlogfuncs.c |    4 +-
src/backend/replication/basebackup.c   |   41 ++-
src/backend/replication/walsender.c    |   50 ++-
src/bin/pg_basebackup/pg_basebackup.c  |   23 +-
src/bin/pg_basebackup/pg_receivexlog.c |   99 +++---
src/bin/pg_basebackup/receivelog.c     |  687 ++++++++++++++++++++++---------
src/bin/pg_basebackup/receivelog.h     |    2 +-
src/include/access/timeline.h          |    3 +-
src/include/access/xlog.h              |    6 +-
12 files changed, 677 insertions(+), 290 deletions(-)


pgsql-committers by date:

Previous
From: Tom Lane
Date:
Subject: pgsql: Improve memory space management in tuplesort and tuplestore.
Next
From: Alvaro Herrera
Date:
Subject: pgsql: Accelerate end-of-transaction dropping of relations