Re: WIP: WAL prefetch (another approach) - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: WIP: WAL prefetch (another approach)
Date
Msg-id 20201007005838.lrpsbpckc5b7giwk@development
Whole thread Raw
In response to Re: WIP: WAL prefetch (another approach)  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers
On Thu, Sep 24, 2020 at 11:38:45AM +1200, Thomas Munro wrote:
>On Wed, Sep 9, 2020 at 11:16 AM Tomas Vondra
><tomas.vondra@2ndquadrant.com> wrote:
>> OK, thanks for looking into this. I guess I'll wait for an updated patch
>> before testing this further. The storage has limited capacity so I'd
>> have to either reduce the amount of data/WAL or juggle with the WAL
>> segments somehow. Doesn't seem worth it.
>
>Here's a new WIP version that works for archive-based recovery in my tests.
>
>The main change I have been working on is that there is now just a
>single XLogReaderState, so no more double-reading and double-decoding
>of the WAL.  It provides XLogReadRecord(), as before, but now you can
>also read further ahead with XLogReadAhead().  The user interface is
>much like before, except that the GUCs changed a bit.  They are now:
>
>  recovery_prefetch=on
>  recovery_prefetch_fpw=off
>  wal_decode_buffer_size=256kB
>  maintenance_io_concurrency=10
>
>I recommend setting maintenance_io_concurrency and
>wal_decode_buffer_size much higher than those defaults.
>

I think you've left the original GUC (replaced by the buffer size) in
the postgresql.conf.sample file. Confused me for a bit ;-)

I've done a bit of testing and so far it seems to work with WAL archive,
so I'll do more testing and benchmarking over the next couple days.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 



pgsql-hackers by date:

Previous
From: Greg Nancarrow
Date:
Subject: Re: Parallel INSERT (INTO ... SELECT ...)
Next
From: Michael Paquier
Date:
Subject: Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?