Re: Bgwriter LRU cleaning: we've been going at this all wrong - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: Bgwriter LRU cleaning: we've been going at this all wrong
Date
Msg-id 46819A1C.1020801@enterprisedb.com
Whole thread Raw
In response to Re: Bgwriter LRU cleaning: we've been going at this all wrong  (Greg Smith <gsmith@gregsmith.com>)
Responses Re: Bgwriter LRU cleaning: we've been going at this all wrong  (Greg Smith <gsmith@gregsmith.com>)
List pgsql-hackers
Greg Smith wrote:
> I broke Itagaki-san's patch into two pieces when I was doing the review 
> cleanup on it specifically to make it easier to tinker with this part 
> without losing some of its other neat features.  Heikki, did you do 
> anything with that LRU adjustment patch since I sent it out: 
> http://archives.postgresql.org/pgsql-patches/2007-05/msg00142.php

I like the idea of breaking down the patch into two parts, though I 
didn't like the bitmasked return code stuff in that first patch.

I haven't worked on that patch. I started looking at this, using 
Itagaki's patch as the basis. In fact, as Tom posted his radical idea, I 
was writing down my thoughts on the bgwriter patch:

I think regardless of the details of how bgwriter should work, the
design is going have three parts:

Part 1: Keeping track of how many buffers have been requested by
backends since last bgwriter round.

Part 2: An algorithm to turn that number into desired # of clean buffers
we should have in front of the clock hand. That could include storing
some historic data to use in the calculation.

Part 3: A way to check that we have that many clean buffers in front of
the clock hand. We might not do that exactly, an approximation would be
enough.

Itagaki's patch attached implements part 1 in the obvious way. A trivial 
implementation for part 2 is (desired # of clean buffers) = (buffers
requested since last round). For part 3, we start from current clock 
hand and scan until we've seen/cleaned enough unpinned buffers with 
usage_count = 0, or until we reach bgwriter_lru_percent.

I think we're good with part 1, but I'm sure everyone has their 
favourite idea for 2 and 3. Let's hear them now.

> Unless someone else has a burning desire to implement Tom's idea faster 
> than me, I should be to build this new implementation myself in the next 
> couple of days.  I still have the test environment leftover from the 
> last time I worked on this code, and I think everybody else who could 
> handle this job has more important higher-level things they could be 
> working on instead.

Oh, that would be great! Since you have the test environment ready, can 
you try alternative patches as well as they're proposed?

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: Bgwriter LRU cleaning: we've been going at this all wrong
Next
From: Florian Pflug
Date:
Subject: Re: Bugtraq: Having Fun With PostgreSQL