Re: WIP: WAL prefetch (another approach) - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: WIP: WAL prefetch (another approach)
Date
Msg-id CA+hUKG+oXRpmc=yOM-GCQMt+Q4GPt9dD_idhPhFVf6f=_owwvA@mail.gmail.com
Whole thread Raw
In response to Re: WIP: WAL prefetch (another approach)  (Stephen Frost <sfrost@snowman.net>)
Responses Re: WIP: WAL prefetch (another approach)
List pgsql-hackers
On Sat, Jun 6, 2020 at 12:36 PM Stephen Frost <sfrost@snowman.net> wrote:
> * Tomas Vondra (tomas.vondra@2ndquadrant.com) wrote:
> > I wonder if we can collect some stats to measure how effective the
> > prefetching actually is. Ultimately we want something like cache hit
> > ratio, but we're only preloading into page cache, so we can't easily
> > measure that. Perhaps we could measure I/O timings in redo, though?
>
> That would certainly be interesting, particularly as this optimization
> seems likely to be useful on some platforms (eg, zfs, where the
> filesystem block size is larger than ours..) and less on others
> (traditional systems which have a smaller block size).

I know one way to get information about cache hit ratios without the
page cache fuzz factor: if you combine this patch with Andres's
still-in-development AIO prototype and tell it to use direct IO, you
get the undiluted truth about hits and misses by looking at the
"prefetch" and "skip_hit" columns of the stats view.  I'm hoping to
have a bit more to say about how this patch works as a client of that
new magic soon, but I also don't want to make this dependent on that
(it's mostly orthogonal, apart from the "how deep is the queue" part
which will improve with better information).

FYI I am still trying to reproduce and understand the problem Tomas
reported; more soon.



pgsql-hackers by date:

Previous
From: Etsuro Fujita
Date:
Subject: Re: estimation problems for DISTINCT ON with FDW
Next
From: Etsuro Fujita
Date:
Subject: Re: Asynchronous Append on postgres_fdw nodes.