pgsql: Improve scalability of md.c for large relations. - Mailing list pgsql-committers

From Andres Freund
Subject pgsql: Improve scalability of md.c for large relations.
Date
Msg-id E1bi9aV-0005L1-LQ@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Improve scalability of md.c for large relations.

So far md.c used a linked list of segments. That proved to be a problem
when processing large relations, because every smgr.c/md.c level access
to a page incurred walking through a linked list of all preceding
segments. Thus making accessing pages O(#segments).

Replace the linked list of segments hanging off SMgrRelationData with an
array of opened segments. That allows O(1) access to individual
segments, if they've previously been opened.

Discussion: <20140331101001.GE13135@alap3.anarazel.de>
Reviewed-By: Peter Geoghegan, Tom Lane (in an older version)

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/45e191e3aa62d47a8bc1a33f784286b2051f45cb

Modified Files
--------------
src/backend/storage/smgr/md.c   | 393 ++++++++++++++++++++++------------------
src/backend/storage/smgr/smgr.c |   4 +-
src/include/storage/smgr.h      |   8 +-
3 files changed, 229 insertions(+), 176 deletions(-)


pgsql-committers by date:

Previous
From: Andres Freund
Date:
Subject: pgsql: Fix mdtruncate() to close fd.c handle of deleted segments.
Next
From: Andres Freund
Date:
Subject: pgsql: Faster PageIsVerified() for the all zeroes case.