Re: Two fsync related performance issues? - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: Two fsync related performance issues?
Date
Msg-id CA+hUKGJafPLFuCwP5AJzYqd5_L4urfNdJJe-xjMkKzpN+e+iEQ@mail.gmail.com
Whole thread Raw
In response to Re: Two fsync related performance issues?  (Thomas Munro <thomas.munro@gmail.com>)
Responses Re: Two fsync related performance issues?  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers
On Thu, Sep 3, 2020 at 11:30 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Wed, May 27, 2020 at 12:31 AM Craig Ringer <craig@2ndquadrant.com> wrote:
> > On Tue, 12 May 2020, 08:42 Paul Guo, <pguo@pivotal.io> wrote:
> >> 1. StartupXLOG() does fsync on the whole data directory early in the crash recovery. I'm wondering if we could
skipsome directories (at least the pg_log/, table directories) since wal, etc could ensure consistency. Here is the
relatedcode.
 
> >>
> >>       if (ControlFile->state != DB_SHUTDOWNED &&
> >>           ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
> >>       {
> >>           RemoveTempXlogFiles();
> >>           SyncDataDirectory();
> >>       }

> 4.  In datadir_fsync_fname(), if ParseRelationPath() is able to
> recognise a file as being a relation file, build a FileTag and call
> RegisterSyncRequest() to tell the checkpointer to sync this file as
> part of the end checkpoint (currently the end-of-recovery checkpoint,
> but that could also be relaxed).

For the record, Andres Freund mentioned a few problems with this
off-list and suggested we consider calling Linux syncfs() for each top
level directory that could potentially be on a different filesystem.
That seems like a nice idea to look into.



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Next
From: Noah Misch
Date:
Subject: Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation