Re: WAL prefetch - Mailing list pgsql-hackers

From Andres Freund
Subject Re: WAL prefetch
Date
Msg-id 20180619164823.vvzovo6csvt66ysl@alap3.anarazel.de
Whole thread Raw
In response to Re: WAL prefetch  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
List pgsql-hackers
Hi,

On 2018-06-19 18:41:24 +0200, Tomas Vondra wrote:
> I'm confused. I thought you wanted to prefetch directly to shared buffers,
> so that it also works with direct I/O in the future. But now you suggest to
> use posix_fadvise() to work around the synchronous buffer read limitation. I
> don't follow ...

Well, I have multiple goals. For one I think using prefetching without
any sort of backpressure and mechanism to see which have completed will
result in hard to monitor and random performance. For another I'm
concerned with wasting a significant amount of memory for the OS cache
of all the read data that's guaranteed to never be needed (as we'll
*always* write to the relevant page shortly down the road).  For those
reasons alone I think prefetching just into the OS cache is a bad idea,
and should be rejected.

I also would want something that's more compatible with DIO. But people
pushed back on that, so...  As long as we build something that looks
like a request queue (which my proposal does), it's also something that
can later with some reduced effort be ported onto asynchronous io.

Greetings,

Andres Freund


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: WAL prefetch
Next
From: Andres Freund
Date:
Subject: Re: Fast default stuff versus pg_upgrade