Re: WIP: WAL prefetch (another approach) - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: WIP: WAL prefetch (another approach)
Date
Msg-id 20200803154659.nzf5qzuozts5dujp@development
Whole thread Raw
In response to Re: WIP: WAL prefetch (another approach)  (Thomas Munro <thomas.munro@gmail.com>)
Responses Re: WIP: WAL prefetch (another approach)  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers
On Thu, Jul 02, 2020 at 03:09:29PM +1200, Thomas Munro wrote:
>On Sat, Jun 6, 2020 at 12:36 PM Stephen Frost <sfrost@snowman.net> wrote:
>> * Tomas Vondra (tomas.vondra@2ndquadrant.com) wrote:
>> > I wonder if we can collect some stats to measure how effective the
>> > prefetching actually is. Ultimately we want something like cache hit
>> > ratio, but we're only preloading into page cache, so we can't easily
>> > measure that. Perhaps we could measure I/O timings in redo, though?
>>
>> That would certainly be interesting, particularly as this optimization
>> seems likely to be useful on some platforms (eg, zfs, where the
>> filesystem block size is larger than ours..) and less on others
>> (traditional systems which have a smaller block size).
>
>I know one way to get information about cache hit ratios without the
>page cache fuzz factor: if you combine this patch with Andres's
>still-in-development AIO prototype and tell it to use direct IO, you
>get the undiluted truth about hits and misses by looking at the
>"prefetch" and "skip_hit" columns of the stats view.  I'm hoping to
>have a bit more to say about how this patch works as a client of that
>new magic soon, but I also don't want to make this dependent on that
>(it's mostly orthogonal, apart from the "how deep is the queue" part
>which will improve with better information).
>
>FYI I am still trying to reproduce and understand the problem Tomas
>reported; more soon.

Any luck trying to reproduce thigs? Should I try again and collect some
additional debug info?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)
Next
From: Robert Haas
Date:
Subject: Re: Can a background worker exist without shared memory access for EXEC_BACKEND cases?