Re: fdatasync(2) on macOS - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: fdatasync(2) on macOS
Date
Msg-id 20210115184325.GC8740@momjian.us
Whole thread Raw
In response to Re: fdatasync(2) on macOS  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Fri, Jan 15, 2021 at 12:55:52PM -0500, Tom Lane wrote:
> > So... does this unreleased function flush drive caches?  We know that
> > fsync(2) doesn't, based on Apple's advice[1] for databases hackers to
> > call fcntl(fd, F_FULLSYNC, 0) instead.  We do that.
> 
> pg_test_fsync results make it clear that fdatasync is the same or a shade
> faster than fsync, which is pretty much what you'd expect.  On my
> late-model Macbook Pro:
> 
> Compare file sync methods using two 8kB writes:
> (in wal_sync_method preference order, except fdatasync is Linux's default)
>         open_datasync                     14251.416 ops/sec      70 usecs/op
>         fdatasync                         25345.103 ops/sec      39 usecs/op
>         fsync                             24677.445 ops/sec      41 usecs/op
>         fsync_writethrough                   41.519 ops/sec   24085 usecs/op
>         open_sync                         14188.903 ops/sec      70 usecs/op
> 
> and on an old Mac mini with spinning rust:
> 
> Compare file sync methods using two 8kB writes:
> (in wal_sync_method preference order, except fdatasync is Linux's default)
>         open_datasync                      2536.535 ops/sec     394 usecs/op
>         fdatasync                          4602.192 ops/sec     217 usecs/op
>         fsync                              4600.365 ops/sec     217 usecs/op
>         fsync_writethrough                   12.135 ops/sec   82404 usecs/op
>         open_sync                          2506.674 ops/sec     399 usecs/op
> 
> So it's not a no-op, but on the other hand it's not succeeding in getting
> bits down to the platter.  I'm not inclined to dike it out, but it does
> seem problematic that we're defaulting to open_datasync, which is also
> not getting bits down to the platter.
> 
> I have a vague recollection that we discussed changing the default
> wal_sync_method for Darwin years ago, but I don't recall why we
> didn't pull the trigger.  These results certainly suggest that
> we oughta.

Is this with an SSD?  We used to be able to know something wasn't
flushing to durable storage because magnetic disk was so slow you could
tell from the numbers, but with SSDs, it might be harder to guess. 
Maybe time to use:

    https://brad.livejournal.com/2116715.html
    diskchecker.pl

or find a way to automate that test.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Git, diffs, and patches
Next
From: Andres Freund
Date:
Subject: Re: O(n^2) system calls in RemoveOldXlogFiles()