pgsql: Accelerate end-of-transaction dropping of relations - Mailing list pgsql-committers

From Alvaro Herrera
Subject pgsql: Accelerate end-of-transaction dropping of relations
Date
Msg-id E1TvuwD-0007tK-4p@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Accelerate end-of-transaction dropping of relations

When relations are dropped, at end of transaction we need to remove the
files and clean the buffer pool of buffers containing pages of those
relations.  Previously we would scan the buffer pool once per relation
to clean up buffers.  When there are many relations to drop, the
repeated scans make this process slow; so we now instead pass a list of
relations to drop and scan the pool once, checking each buffer against
the passed list.  When the number of relations is larger than a
threshold (which as of this patch is being set to 20 relations) we sort
the array before starting, and bsearch the array; when it's smaller, we
simply scan the array linearly each time, because that's faster.  The
exact optimal threshold value depends on many factors, but the
difference is not likely to be significant enough to justify making it
user-settable.

This has been measured to be a significant win (a 15x win when dropping
100,000 relations; an extreme case, but reportedly a real one).

Author: Tomas Vondra, some tweaks by me
Reviewed by: Robert Haas, Shigeru Hanada, Andres Freund, Álvaro Herrera

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/279628a0a7cf582f7dfb68e25b7b76183dd8ff2f

Modified Files
--------------
src/backend/catalog/storage.c       |   26 ++++++++-
src/backend/storage/buffer/bufmgr.c |  109 +++++++++++++++++++++++++++++++---
src/backend/storage/smgr/smgr.c     |   82 ++++++++++++++++++++++++++-
src/include/storage/bufmgr.h        |    2 +-
src/include/storage/smgr.h          |    1 +
5 files changed, 206 insertions(+), 14 deletions(-)


pgsql-committers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: pgsql: Make pg_receivexlog and pg_basebackup -X stream work across time
Next
From: Heikki Linnakangas
Date:
Subject: pgsql: When xlogreader asks the callback function to read a page, make