Re: Bug: Buffer cache is not scan resistant - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: Bug: Buffer cache is not scan resistant
Date
Msg-id 1173203763.13722.414.camel@dogma.v10.wvs
Whole thread Raw
In response to Re: Bug: Buffer cache is not scan resistant  (Jim Nasby <decibel@decibel.org>)
Responses Re: Bug: Buffer cache is not scan resistant
Re: Bug: Buffer cache is not scan resistant
Re: Bug: Buffer cache is not scan resistant
List pgsql-hackers
On Mon, 2007-03-05 at 21:02 -0700, Jim Nasby wrote:
> On Mar 5, 2007, at 2:03 PM, Heikki Linnakangas wrote:
> > Another approach I proposed back in December is to not have a  
> > variable like that at all, but scan the buffer cache for pages  
> > belonging to the table you're scanning to initialize the scan.  
> > Scanning all the BufferDescs is a fairly CPU and lock heavy  
> > operation, but it might be ok given that we're talking about large  
> > I/O bound sequential scans. It would require no DBA tuning and  
> > would work more robustly in varying conditions. I'm not sure where  
> > you would continue after scanning the in-cache pages. At the  
> > highest in-cache block number, perhaps.
> 
> If there was some way to do that, it'd be what I'd vote for.
> 

I still don't know how to make this take advantage of the OS buffer
cache. 

However, no DBA tuning is a huge advantage, I agree with that.

If I were to implement this idea, I think Heikki's bitmap of pages
already read is the way to go. Can you guys give me some pointers about
how to walk through the shared buffers, reading the pages that I need,
while being sure not to read a page that's been evicted, and also not
potentially causing a performance regression somewhere else?

> Given the partitioning of the buffer lock that Tom did it might not  
> be that horrible for many cases, either, since you'd only need to  
> scan through one partition.
> 
> We also don't need an exact count, either. Perhaps there's some way  
> we could keep a counter or something...

Exact count of what? The pages already in cache?

Regards,Jeff Davis



pgsql-hackers by date:

Previous
From: Gregory Stark
Date:
Subject: Re: GIST and TOAST
Next
From: Tom Lane
Date:
Subject: Re: Bug: Buffer cache is not scan resistant