Re: Bgwriter LRU cleaning: we've been going at this all wrong - Mailing list pgsql-hackers

From ITAGAKI Takahiro
Subject Re: Bgwriter LRU cleaning: we've been going at this all wrong
Date
Msg-id 20070627110859.6410.ITAGAKI.TAKAHIRO@oss.ntt.co.jp
Whole thread Raw
In response to Re: Bgwriter LRU cleaning: we've been going at this all wrong  (Heikki Linnakangas <heikki@enterprisedb.com>)
Responses Re: Bgwriter LRU cleaning: we've been going at this all wrong
List pgsql-hackers
Heikki Linnakangas <heikki@enterprisedb.com> wrote:

> Tom Lane wrote:
> > In fact, the notion of the bgwriter's cleaning scan being "in front of"
> > the clock sweep is entirely backward.  It should try to be behind the
> > sweep, ie, so far ahead that it's lapped the clock sweep and is trailing
> > along right behind it, cleaning buffers immediately after their
> > usage_count falls to zero.  All the rest of the buffer arena is either
> > clean or has positive usage_count.
>
> That will vary widely depending on your workload, of course, but keeping 
> 1/4 of the buffer cache clean seems like overkill to me. If any of those 
> buffers are re-dirtied after we write them, the write was a waste of time.

Agreed intuitively, but I don't know how offen backends change usage_count
0 to 1. If the rate is high, backward-bgwriter would not work. It seems to
happen frequently when we use large shared buffers.

I read Tom is changing the bgwriter LRU policy from "clean dirty pages
recycled soon" to "clean dirty pages just when they turn out to be less
frequently used", right? I have another thought -- advancing bgwriter's
sweep-startpoint a little ahead. 

[buf] 0    lru     X    bgw-start                      N     |-----|----------->|-----------------------------|

I think X=0 is in the current behavior and X=N is in the backward-bgwriter.
Are there any other appropriate values for X? It might be good to use
statistics information about buffer usage to modify X runtime.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center




pgsql-hackers by date:

Previous
From: Jeremy Drake
Date:
Subject: Re: Bugtraq: Having Fun With PostgreSQL
Next
From: "Andrew Dunstan"
Date:
Subject: Re: Bugtraq: Having Fun With PostgreSQL