Re: Bug: Buffer cache is not scan resistant - Mailing list pgsql-hackers

<p><font size="2">Cool!<br /><br /> - Luke<br /><br /> Msg is shrt cuz m on ma treo<br /><br />  -----Original
Message-----<br/> From:   Simon Riggs [<a href="mailto:simon@2ndquadrant.com">mailto:simon@2ndquadrant.com</a>]<br />
Sent:  Friday, March 09, 2007 02:32 PM Eastern Standard Time<br /> To:     Luke Lonergan; ITAGAKI Takahiro<br />
Cc:    Sherry Moore; Tom Lane; Mark Kirkwood; Pavan Deolasee; Gavin Sherry; PGSQL Hackers; Doug Rady<br />
Subject:       Re: [HACKERS] Bug: Buffer cache is not scan resistant<br /><br /> On Tue, 2007-03-06 at 22:32 -0500,
LukeLonergan wrote:<br /> > Incidentally, we tried triggering NTA (L2 cache bypass)<br /> > unconditionally and
invarious patterns and did not see the<br /> > substantial gain as with reducing the working set size.<br /> ><br
/>> My conclusion: Fixing the OS is not sufficient to alleviate the issue.<br /> > We see a 2x penalty (1700MB/s
versus3500MB/s) at the higher data<br /> > rates due to this effect.<br /> ><br /> I've implemented buffer
recycling,as previously described, patch being<br /> posted now to -patches as "scan_recycle_buffers".<br /><br /> This
versionincludes buffer recycling<br /><br /> - for SeqScans larger than shared buffers, with the objective of<br />
improvingL2 cache efficiency *and* reducing the effects of shared<br /> buffer cache spoiling (both as previously
discussedon this thread)<br /><br /> - for VACUUMs of any size, with the objective of reducing WAL thrashing<br />
whilstkeeping VACUUM's behaviour of not spoiling the buffer cache (as<br /> originally suggested by Itagaki-san, just
witha different<br /> implementation).<br /><br /> Behaviour is not activated by default in this patch. To request
buffer<br/> recycling, set the USERSET GUC<br />         SET scan_recycle_buffers = N<br /> tested with 1,4,8,16, but
only> 8 seems sensible, IMHO.<br /><br /> Patch effects StrategyGetBuffer, so only effects the disk->cache
path.<br/> The idea is that if its already in shared buffer cache then we get<br /> substantial benefit already and
nothingelse is needed. So for the<br /> general case, the patch adds a single if test into the I/O path.<br /><br />
Theparameter is picked up at the start of SeqScan and VACUUM<br /> (currently). Any change mid-scan will be ignored.<br
/><br/> IMHO its possible to do this and to allow Synch Scans at the same time,<br /> with some thought. There is no
needfor us to rely on cache spoiling<br /> behaviour of scans to implement that feature as well.<br /><br />
Independentperformance tests requested, so that we can discuss this<br /> objectively.<br /><br /> --<br />   Simon
Riggs            <br/>   EnterpriseDB   <a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a><br /><br
/><br/><br /></font>