Re: Use fadvise in wal replay - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: Use fadvise in wal replay
Date
Msg-id 96C1BADB-362B-4C02-8799-0118B44C2025@yandex-team.ru
Whole thread Raw
In response to Re: Use fadvise in wal replay  (Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>)
Responses Re: Use fadvise in wal replay
List pgsql-hackers

> On 7 Aug 2022, at 06:39, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
>
> Agree. Why can't we just prefetch the entire WAL file once whenever it
> is opened for the first time? Does the OS have any limitations on max
> size to prefetch at once? It may sound aggressive, but it avoids
> fadvise() system calls, this will be especially useful if there are
> many WAL files to recover (crash, PITR or standby recovery),
> eventually we would want the total WAL file to be prefetched.
>
> If prefetching the entire WAL file is okay, we could further do this:
> 1) prefetch in XLogFileOpen() and all of segment_open callbacks, 2)
> release in XLogFileClose (it's being dong right now) and all of
> segment_close callbacks - do this perhaps optionally.
>
> Also, can't we use an existing function FilePrefetch()? That way,
> there is no need for a new wait event type.
>
> Thoughts?

Thomas expressed this idea upthread. Benchmarks done by Jakub showed that this approach had no significant improvement
overexisting master code. 
The same benchmarks showed almost x1.5 improvement of readahead in 8Kb or 128Kb chunks.

Best regards, Andrey Borodin.


pgsql-hackers by date:

Previous
From: Bharath Rupireddy
Date:
Subject: Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
Next
From: Andres Freund
Date:
Subject: Re: bug on log generation ?