Re: Bug: Buffer cache is not scan resistant - Mailing list pgsql-hackers

From Josh Berkus
Subject Re: Bug: Buffer cache is not scan resistant
Date
Msg-id 200703051046.16281.josh@agliodbs.com
Whole thread Raw
In response to Re: Bug: Buffer cache is not scan resistant  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Bug: Buffer cache is not scan resistant  ("Simon Riggs" <simon@2ndquadrant.com>)
Re: Bug: Buffer cache is not scan resistant  (Jim Nasby <decibel@decibel.org>)
List pgsql-hackers
Tom,

> I seem to recall that we've previously discussed the idea of letting the
> clock sweep decrement the usage_count before testing for 0, so that a
> buffer could be reused on the first sweep after it was initially used,
> but that we rejected it as being a bad idea.  But at least with large
> shared_buffers it doesn't sound like such a bad idea.

We did discuss an number of formulas for setting buffers with different
clock-sweep numbers, including ones with higher usage_count for indexes and
starting numbers of 0 for large seq scans as well as vacuums.  However, we
didn't have any way to prove that any of these complex algorithms would
result in higher performance, so went with the simplest formula, with the
idea of tinkering with it when we had more data.  So maybe now's the time.

Note, though, that the current algorithm is working very, very well for OLTP
benchmarks, so we'd want to be careful not to gain performance in one area at
the expense of another.  In TPCE testing, we've been able to increase
shared_buffers to 10GB with beneficial performance effect (numbers posted
when I have them) and even found that "taking over RAM" with the
shared_buffers (ala Oracle) gave us equivalent performance to using the FS
cache.  (yes, this means with a little I/O management engineering we could
contemplate discarding use of the FS cache for a net performance gain.  Maybe
for 8.4)

--
Josh Berkus
PostgreSQL @ Sun
San Francisco


pgsql-hackers by date:

Previous
From: "Luke Lonergan"
Date:
Subject: Re: Bug: Buffer cache is not scan resistant
Next
From: "Pavan Deolasee"
Date:
Subject: Re: Bug: Buffer cache is not scan resistant