Re: Blocking I/O, async I/O and io_uring - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: Blocking I/O, async I/O and io_uring
Date
Msg-id CA+hUKGKwArCVb=rdv252yX0GrzkiS+vw7ExAjK7O0bJDUkfzJQ@mail.gmail.com
Whole thread Raw
In response to Blocking I/O, async I/O and io_uring  (Craig Ringer <craig.ringer@enterprisedb.com>)
List pgsql-hackers
On Tue, Dec 8, 2020 at 3:56 PM Craig Ringer
<craig.ringer@enterprisedb.com> wrote:
> I thought I'd start the discussion on this and see where we can go with it. What incremental steps can be done to
moveus toward parallelisable I/O without having to redesign everything? 
>
> I'm thinking that redo is probably a good first candidate. It doesn't depend on the guts of the executor. It is much
lesssensitive to ordering between operations in shmem and on disk since it runs in the startup process. And it hurts
REALLYBADLY from its single-threaded blocking approach to I/O - as shown by an extension written by 2ndQuadrant that
candouble redo performance by doing read-ahead on btree pages that will soon be needed. 

About the redo suggestion: https://commitfest.postgresql.org/31/2410/
does exactly that!  It currently uses POSIX_FADV_WILLNEED because
that's what PrefetchSharedBuffer() does, but when combined with a
"real AIO" patch set (see earlier threads and conference talks on this
by Andres) and a few small tweaks to control batching of I/O
submissions, it does exactly what you're describing.  I tried to keep
the WAL prefetcher project entirely disentangled from the core AIO
work, though, hence the "poor man's AIO" for now.



pgsql-hackers by date:

Previous
From: "tsunakawa.takay@fujitsu.com"
Date:
Subject: RE: [bug fix] ALTER TABLE SET LOGGED/UNLOGGED on a partitioned table does nothing silently
Next
From: Michael Paquier
Date:
Subject: Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.