Re: 2nd Level Buffer Cache - Mailing list pgsql-hackers

From Greg Smith
Subject Re: 2nd Level Buffer Cache
Date
Msg-id 4D951125.3030206@2ndQuadrant.com
Whole thread Raw
In response to Re: 2nd Level Buffer Cache  (Jim Nasby <jim@nasby.net>)
List pgsql-hackers
On 03/24/2011 03:36 PM, Jim Nasby wrote:
> On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
>    
>> Robert Haas<robertmhaas@gmail.com>  writes:
>>      
>>> It looks like the only way anything can ever get put on the free list
>>> right now is if a relation or database is dropped.  That doesn't seem
>>> too good.
>>>        
>> Why not?  AIUI the free list is only for buffers that are totally dead,
>> ie contain no info that's possibly of interest to anybody.  It is *not*
>> meant to substitute for running the clock sweep when you have to discard
>> a live buffer.
>>      
> Turns out we've had this discussion before: http://archives.postgresql.org/pgsql-hackers/2010-12/msg01088.php and
http://archives.postgresql.org/pgsql-hackers/2010-12/msg00689.php
>    

Investigating this has been on the TODO list for four years now:

http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php

I feel that work in this area is blocked behind putting together a 
decent mix of benchmarks that can be used to test whether changes here 
are actually good or bad.  All of the easy changes to buffer allocation 
strategy, ones that you could verify by inspection and simple tests, 
were made in 8.3.  The stuff that's left has the potential to either 
improve or reduce performance, and which will happen is very workload 
dependent.

Setting up systematic benchmarks of multiple workloads to run 
continuously on big hardware is a large, boring, expensive problem that 
few can justify financing (except for Jim of course), and even fewer 
want to volunteer time toward.  This whole discussion of cache policy 
tweaks is fun, but I just delete all the discussion now because it's 
just going in circles without a good testing regime.  The right way to 
start is by saying "this is the benchmark I'm going to improve with this 
change, and it has a profiled hotspot at this point".

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books



pgsql-hackers by date:

Previous
From: "Joshua D. Drake"
Date:
Subject: Re: Should psql support URI syntax?
Next
From: Thom Brown
Date:
Subject: Foreign table permissions and cloning