Re: postgresql latency & bgwriter not doing its job - Mailing list pgsql-hackers

From Tom Lane
Subject Re: postgresql latency & bgwriter not doing its job
Date
Msg-id 6599.1409421040@sss.pgh.pa.us
Whole thread Raw
In response to Re: postgresql latency & bgwriter not doing its job  (Andres Freund <andres@2ndquadrant.com>)
Responses Re: postgresql latency & bgwriter not doing its job
Re: postgresql latency & bgwriter not doing its job
List pgsql-hackers
Andres Freund <andres@2ndquadrant.com> writes:
> On 2014-08-27 19:23:04 +0300, Heikki Linnakangas wrote:
>> A long time ago, Itagaki Takahiro wrote a patch sort the buffers and write
>> them out in order (http://www.postgresql.org/message-id/flat/20070614153758.6A62.ITAGAKI.TAKAHIRO@oss.ntt.co.jp).
>> The performance impact of that was inconclusive, but one thing that it
>> allows nicely is to interleave the fsyncs, so that you write all the buffers
>> for one file, then fsync it, then next file and so on.

> ...
> So, *very* clearly sorting is a benefit.

pg_bench alone doesn't convince me on this.  The original thread found
cases where it was a loss, IIRC; you will need to test many more than
one scenario to prove the point.

Also, it does not matter how good it looks in test cases if it causes
outright failures due to OOM; unlike you, I am not prepared to just "wave
away" that risk.  A possible compromise is to sort a limited number of
buffers ---- say, collect a few thousand dirty buffers then sort, dump and
fsync them, repeat as needed.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Built-in binning functions
Next
From: Andres Freund
Date:
Subject: Re: postgresql latency & bgwriter not doing its job