Re: proposal: be smarter about i/o patterns in index scan - Mailing list pgsql-hackers

From Tom Lane
Subject Re: proposal: be smarter about i/o patterns in index scan
Date
Msg-id 22096.1085008156@sss.pgh.pa.us
Whole thread Raw
In response to Re: proposal: be smarter about i/o patterns in index scan  ("Jeffrey W. Baker" <jwbaker@acm.org>)
Responses Re: proposal: be smarter about i/o patterns in index scan  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-hackers
"Jeffrey W. Baker" <jwbaker@acm.org> writes:
> ... Introducing a new query execution step sounds
> like a better/easier idea than was I was going to do, which was to
> rework some of the access methods to operate on vectors instead of
> scalars.  That idea looks increasingly difficult to implement.

One thing that I think is worth doing in any case is to alter the API
for the index AMs' getnext functions so that they can return multiple
TIDs per call (probably into an array supplied by the caller).  We could
for example return all matching tuples from a single index page in one
call, and then let index_getnext iterate through them without re-calling
the index AM.  (Making it work per-index-page would allow us to keep the
current VACUUM interlocking conventions in place exactly, so that there
would be no risk of breaking anything.  All tuples returned in a given
call would still be protected by an index-page lock.)

We'd have to make such an API change anyway to support unordered
indexscan, but it should be a benefit even for ordered scans, because
it should considerably reduce the locking overhead for indexscans that
fetch multiple tuples.  In particular it might alleviate the BufMgrLock
contention problems that were under discussion last month.  (The test
case we were using to illustrate that problem fetched several hundred
tuples per indexscan, so it clearly could benefit.  Extent of benefit
unknown at this time, though.)

The tricky part of this is figuring out how to handle mark/restore and
scan direction switching in a way that doesn't complicate the code to
the point of unmaintainability.  I think it may be possible to keep all
the extra complication within indexam.c, but haven't thought through the
details.

This seems like a relatively localized change, and might be a good place
for you to start.
        regards, tom lane


pgsql-hackers by date:

Previous
From: pgsql@mohawksoft.com
Date:
Subject: Re: Table Spaces
Next
From: "Andrew Dunstan"
Date:
Subject: Re: Table Spaces