Re: Improvement of checkpoint IO scheduler for stable transaction responses - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: Improvement of checkpoint IO scheduler for stable transaction responses
Date
Msg-id 51EAE6F1.1010402@vmware.com
Whole thread Raw
In response to Re: Improvement of checkpoint IO scheduler for stable transaction responses  (Greg Smith <greg@2ndQuadrant.com>)
Responses Re: Improvement of checkpoint IO scheduler for stable transaction responses  (KONDO Mitsumasa <kondo.mitsumasa@lab.ntt.co.jp>)
List pgsql-hackers
On 20.07.2013 19:28, Greg Smith wrote:
> On 7/20/13 4:48 AM, didier wrote:
>> With your tests did you try to write the hot buffers first? ie buffers
>> with a high refcount, either by sorting them on refcount or at least
>> sweeping the buffer list in reverse?
>
> I never tried that version. After a few rounds of seeing that all
> changes I tried were just rearranging the good and bad cases, I got
> pretty bored with trying new changes in that same style.

It doesn't seem like we're getting anywhere with minor changes to the
existing logic. The reason I brought up sorting the writes in the first
place is that it allows you to fsync() each segment after it's written,
rather than doing all the writes first, and then fsyncing all the relations.

Mitsumasa-san, since you have the test rig ready, could you try the
attached patch please? It scans the buffer cache several times, writing
out all the dirty buffers for segment A first, then fsyncs it, then all
dirty buffers for segment B, and so on. The patch is ugly, but if it
proves to be helpful, we can spend the time to clean it up.

- Heikki

Attachment

pgsql-hackers by date:

Previous
From: Greg Smith
Date:
Subject: Re: Improvement of checkpoint IO scheduler for stable transaction responses
Next
From: Noah Misch
Date:
Subject: InvokeObjectPostAlterHook() vs. CommandCounterIncrement()