Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS
Date
Msg-id CAH2-Wz=y_K7t2KtGAm1o31okhdYb5+Jt=hV4V0F2nxSgJysAAw@mail.gmail.com
Whole thread Raw
In response to Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Mon, Apr 2, 2018 at 7:54 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> Also, this really does make it impossible to write reliable programs.
> Imagine that, while the server is running, somebody runs a program
> which opens a file in the data directory, calls fsync() on it, and
> closes it.  If the fsync() fails, postgres is now borked and has no
> way of being aware of the problem.  If we knew, we could PANIC, but
> we'll never find out, because the unrelated process ate the error.
> This is exactly the sort of ill-considered behavior that makes fcntl()
> locking nearly useless.

I fear that the conventional wisdom from the Kernel people is now "you
should be using O_DIRECT for granular control".  The LWN article
Thomas linked (https://lwn.net/Articles/718734) cites Ted Ts'o:

"Monakhov asked why a counter was needed; Layton said it was to handle
multiple overlapping writebacks. Effectively, the counter would record
whether a writeback had failed since the file was opened or since the
last fsync(). Ts'o said that should be fine; applications that want
more information should use O_DIRECT. For most applications, knowledge
that an error occurred somewhere in the file is all that is necessary;
applications that require better granularity already use O_DIRECT."

-- 
Peter Geoghegan


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [HACKERS] MERGE SQL Statement for PG11
Next
From: Peter Geoghegan
Date:
Subject: Re: WIP: Covering + unique indexes.