Re: initdb and fsync - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: initdb and fsync
Date
Msg-id 1331610580.6425.98.camel@jdavis
Whole thread Raw
In response to Re: initdb and fsync  (Noah Misch <noah@leadboat.com>)
Responses Re: initdb and fsync  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Sun, 2012-02-05 at 17:56 -0500, Noah Misch wrote:
> I meant primarily to illustrate the need to be comprehensive, not comment on
> which executable should fsync a particular file.  Bootstrap-mode backends do
> not sync anything during an initdb run on my system.  With your patch, we'll
> fsync a small handful of files and leave nearly everything else vulnerable.

Thank you for pointing that out. With that in mind, I have a new version
of the patch which just recursively fsync's the whole directory
(attached).

I also introduced a new option --nosync (-N) to disable this behavior.

The bad news is that it introduces a lot more time to initdb -- it goes
from about 1s to about 10s on my machine. I tried fsync'ing the whole
directory twice just to make sure that the second was a no-op, and
indeed it didn't make much difference (still about 10s).

That's pretty inefficient considering that

  initdb -D data --nosync && sync

only takes a couple seconds. Clearly batching the operation is a big
help. Maybe there's some more efficient way to fsync a lot of
files/directories? Or maybe I can mitigate it by avoiding files that
don't really need to be fsync'd?

Regards,
    Jeff Davis

Attachment

pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Re: xlog location arithmetic
Next
From: Tom Lane
Date:
Subject: Re: pg_upgrade and statistics