pg_start_backup without checkpoint patch (a part of Synch Rep) - Mailing list pgsql-hackers

From Fujii Masao
Subject pg_start_backup without checkpoint patch (a part of Synch Rep)
Date
Msg-id 3f0b79eb0812270428y1f6f2422ve940caab53d63142@mail.gmail.com
Whole thread Raw
Responses Re: pg_start_backup without checkpoint patch (a part of Synch Rep)  (Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>)
List pgsql-hackers
Hi,

Attached is the self-contained patch to skip checkpoint at pg_start_backup.
This is a part of Synch Rep patches, and was discussed in the following
thread.
http://archives.postgresql.org/message-id/3f0b79eb0812240710j7e613f3atfd6b6fc27403546e@mail.gmail.com

In Synch Rep, we basically have to get a fresh backup, to make the failed
server catch up with the primary after failover. But getting an online-backup
is expensive operation because of pg_start_backup's checkpoint and data
copying. Especially since pg_start_backup is performed as one transaction,
its checkpoint might not only increase I/O traffic but also cause
long-transaction which prevents VACUUM and HOT. This patch makes
pg_start_backup skip a checkpoint if possible, and works out the above
problem.

Specifically, pg_start_backup uses the last checkpoint instead of doing a
new checkpoint if full_page_writes = on since the last checkpoint, which
guarantees that all the full-pages required for PITR are written.

There is one constraint: XLR_BKP_REMOVABLE flag cannot be marked
correctly, because we cannot judge whether the full-pages (generated
before pg_start_backup) are removal or not. AFAIK, this flag is only used
by pglesslog. Should we add a new option to specify whether checkpoint
is skippable, for the backward compatibility?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment

pgsql-hackers by date:

Previous
From: Gregory Stark
Date:
Subject: Re: V3 of PITR performance improvement for 8.4 (WIP)
Next
From: "Hitoshi Harada"
Date:
Subject: Re: Tuplestore trimming in window-functions patch