Re: Spread checkpoint sync - Mailing list pgsql-hackers

Robert Haas wrote: <blockquote cite="mid:AANLkTi=P0te3oFq0LVS8cGLkGF_Wp9ery0fOu9SHEcs9@mail.gmail.com"
type="cite"><prewrap="">On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith <a class="moz-txt-link-rfc2396E"
href="mailto:greg@2ndquadrant.com"><greg@2ndquadrant.com></a>wrote: </pre><blockquote type="cite"><pre
wrap="">Oneof the ideas Simon and I had been considering at one point was adding
 
some better de-duplication logic to the fsync absorb code, which I'm
reminded by the pattern here might be helpful independently of other
improvements.   </pre></blockquote><pre wrap="">
Hopefully I'm not stepping on any toes here, but I thought this was an
awfully good idea and had a chance to take a look at how hard it would
be today while en route from point A to point B.  The answer turned
out to be "not very", so PFA a patch that seems to work.  I tested it
by attaching gdb to the background writer while running pgbench, and
it eliminate the backend fsyncs without even breaking a sweat. </pre></blockquote><br /> No toe damage, this is great,
Ihadn't gotten to coding for this angle yet at all.  Suffering from an overload of ideas and (mostly wasted) test data,
sothanks for exploring this concept and proving it works.<br /><br /> I'm not sure what to do with the rest of the work
I'vebeen doing in this area here, so I'm tempted to just combine this new bit from you with the older patch I
submitted,streamline, and see if that makes sense.  Expected to be there already, then "how about spending 5 minutes
firstchecking out that autovacuum lock patch again" turned out to be a wild underestimate.<br /><br /> Part of the
problemis that it's become obvious to me the last month that right now is a terrible time to be doing Linux benchmarks
thatimpact filesystem sync behavior.  The recent kernel changes that are showing in the next rev of the enterprise
distributions--likeRHEL6 and Debian Squeeze both working to get a stable 2.6.32--have made testing a nightmare.  I
don'twant to dump a lot of time into optimizing for <2.6.32 if this problem changes its form in newer kernels, but
thedistributions built around newer kernels are just not fully baked enough yet to tell.  For example, the pre-release
Squeezenumbers we're seeing are awful so far, but it's not really done yet either.  I expect 3-6 months from today,
thatall will have settled down enough that I can make some sense of it.  Lately my work with the latest distributions
hasjust been burning time installing stuff that doesn't work quite right yet.<br /><br /><pre class="moz-signature"
cols="72">--
 
Greg Smith   2ndQuadrant US    <a class="moz-txt-link-abbreviated"
href="mailto:greg@2ndQuadrant.com">greg@2ndQuadrant.com</a>  Baltimore, MD
 
PostgreSQL Training, Services, and 24x7 Support  <a class="moz-txt-link-abbreviated"
href="http://www.2ndQuadrant.us">www.2ndQuadrant.us</a>
"PostgreSQL 9.0 High Performance": <a class="moz-txt-link-freetext"
href="http://www.2ndQuadrant.com/books">http://www.2ndQuadrant.com/books</a>
</pre>