Re: Experimental patch for inter-page delay in VACUUM - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Experimental patch for inter-page delay in VACUUM
Date
Msg-id 22712.1067964543@sss.pgh.pa.us
Whole thread Raw
In response to Re: Experimental patch for inter-page delay in VACUUM  (Jan Wieck <JanWieck@Yahoo.com>)
Responses Re: Experimental patch for inter-page delay in VACUUM  (Jan Wieck <JanWieck@Yahoo.com>)
Re: Experimental patch for inter-page delay in VACUUM  (Bruce Momjian <pgman@candle.pha.pa.us>)
List pgsql-hackers
Jan Wieck <JanWieck@Yahoo.com> writes:
> That is part of the idea. The whole idea is to issue "physical" writes 
> at a fairly steady rate without increasing the number of them 
> substantial or interfering with the drives opinion about their order too 
> much. I think O_SYNC for random access can be in conflict with write 
> reordering.

Good point.  But if we issue lots of writes without fsync then we still
have the problem of a write storm when the fsync finally occurs, while
if we fsync too often then we constrain the write order too much.  There
will need to be some tuning here.

> How I can see the background writer operating is that he's keeping the 
> buffers in the order of the LRU chain(s) clean, because those are the 
> buffers that most likely get replaced soon. In my experimental ARC code 
> it would traverse the T1 and T2 queues from LRU to MRU, write out n1 and 
> n2 dirty buffers (n1+n2 configurable), then fsync all files that have 
> been involved in that, nap depending on where he got down the queues (to 
> increase the write rate when running low on clean buffers), and do it 
> all over again.

You probably need one more knob here: how often to issue the fsyncs.
I'm not convinced "once per outer loop" is a sufficient answer.
Otherwise this is sounding pretty good.
        regards, tom lane


pgsql-hackers by date:

Previous
From: "Stephen"
Date:
Subject: Re: Experimental patch for inter-page delay in VACUUM
Next
From: "Carroll, Catherine A."
Date:
Subject: Re: [DOCS] Annotated release notes