Re: WAL prefetch - Mailing list pgsql-hackers

From Andres Freund
Subject Re: WAL prefetch
Date
Msg-id 20180618204741.yf6qw445pe6hlsqn@alap3.anarazel.de
Whole thread Raw
In response to Re: WAL prefetch  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: WAL prefetch
List pgsql-hackers
On 2018-06-18 16:44:09 -0400, Robert Haas wrote:
> On Sat, Jun 16, 2018 at 3:41 PM, Andres Freund <andres@anarazel.de> wrote:
> >> The posix_fadvise approach is not perfect, no doubt about that. But it
> >> works pretty well for bitmap heap scans, and it's about 13249x better
> >> (rough estimate) than the current solution (no prefetching).
> >
> > Sure, but investing in an architecture we know might not live long also
> > has it's cost. Especially if it's not that complicated to do better.
> 
> My guesses are:
> 
> - Using OS prefetching is a very small patch.
> - Prefetching into shared buffers is a much bigger patch.

Why?  The majority of the work is standing up a bgworker that does
prefetching (i.e. reads WAL, figures out reads not in s_b, does
prefetch). Allowing a configurable number + some synchronization between
them isn't that much more work.


> - It'll be five years before we have direct I/O.

I think we'll have lost a significant market share by then if that's the
case. Deservedly so.

Greetings,

Andres Freund


pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: WAL prefetch
Next
From: Alvaro Herrera
Date:
Subject: Re: [HACKERS] Statement-level rollback