pgsql: Improve performance of pgarch_readyXlog() with many status files - Mailing list pgsql-committers

From Robert Haas
Subject pgsql: Improve performance of pgarch_readyXlog() with many status files
Date
Msg-id E1mlGa2-0001A1-HJ@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Improve performance of pgarch_readyXlog() with many status files.

Presently, the archive_status directory was scanned for each file to
archive.  When there are many status files, say because archive_command
has been failing for a long time, these directory scans can get very
slow.  With this change, the archiver remembers several files to archive
during each directory scan, speeding things up.

To ensure timeline history files are archived as quickly as possible,
XLogArchiveNotify() forces the archiver to do a new directory scan as
soon as the .ready file for one is created.

Nathan Bossart, per a long discussion involving many people. It is
not clear to me exactly who out of all those people reviewed this
particular patch.

Discussion: http://postgr.es/m/CA+TgmobhAbs2yabTuTRkJTq_kkC80-+jw=pfpypdOJ7+gAbQbw@mail.gmail.com
Discussion: http://postgr.es/m/620F3CE1-0255-4D66-9D87-0EADE866985A@amazon.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/beb4e9ba1652a04f66ff20261444d06f678c0b2d

Modified Files
--------------
src/backend/access/transam/xlogarchive.c |  14 +++
src/backend/postmaster/pgarch.c          | 193 ++++++++++++++++++++++++++-----
src/include/postmaster/pgarch.h          |   1 +
3 files changed, 182 insertions(+), 26 deletions(-)


pgsql-committers by date:

Previous
From: Tom Lane
Date:
Subject: pgsql: Fall back to unsigned int, not int, for socklen_t.
Next
From: Peter Geoghegan
Date:
Subject: pgsql: Update another obsolete reference in vacuumlazy.c.