Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb) - Mailing list pgsql-hackers

From Andres Freund
Subject Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Date
Msg-id 200912282354.51892.andres@anarazel.de
Whole thread Raw
Responses Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Andres Freund <andres@anarazel.de>)
Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)  (Greg Stark <gsstark@mit.edu>)
List pgsql-hackers
On Saturday 12 December 2009 21:38:41 Andres Freund wrote:
> On Saturday 12 December 2009 21:36:27 Michael Clemmons wrote:
> > If ppl think its worth it I'll create a ticket
> Thanks, no need. I will post a patch tomorrow or so.
Well. It was a long day...

Anyway.
In this patch I delay the fsync done in copy_file and simply do a second pass
over the directory in copy_dir and fsync everything in that pass.
Including the directory - which was not done before and actually might be
necessary in some cases.
I added a posix_fadvise(..., FADV_DONTNEED) to make it more likely that the
copied file reaches storage before the fsync. Without the speed benefits were
quite a bit smaller and essentially random (which seems sensible).

This speeds up CREATE DATABASE from ~9 seconds to something around 0.8s on my
laptop.  Still slower than with fsync off (~0.25) but quite a worthy
improvement.

The benefits are obviously bigger if the template database includes anything
added.


Andres

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Stats for inheritance trees
Next
From: Dimitri Fontaine
Date:
Subject: Re: Admission Control Policy