Re: Is there a way to speed up WAL replay? - Mailing list pgsql-general

From Stephen Frost
Subject Re: Is there a way to speed up WAL replay?
Date
Msg-id 20181031153252.GY4184@tamriel.snowman.net
Whole thread Raw
In response to Re: Is there a way to speed up WAL replay?  (Jeff Janes <jeff.janes@gmail.com>)
List pgsql-general
Greetings,

* Jeff Janes (jeff.janes@gmail.com) wrote:
> One way I found to speed up restore_command is to have another program run
> a few WAL files ahead of it, copying the WAL from the real archive into a
> scratch space which is on the same filesystem as pg_xlog/pg_wal.  Then have
> restore_command simply move (not copy) the requested files from the scratch
> space onto %p.  The intra-filesystem move completes much faster than a copy.

Right, that can definitely be a big help and is more-or-less what
pgbackrest does too.

> If it spends time waiting for blocks that need to be recovered into to get
> read from disk, and you have enough RAM, you could speed it up by
> pre-warming the file system cache.  Something like:
>
> tar -cf - $PGDATA | wc -c

Yeah, that's also a good idea.  We've discussed having something in
pgbackrest to basically go pre-load things off disk in advance of WAL
replay, and I seem to recall someone had also written an external tool
to do that.

Thanks!

Stephen

Attachment

pgsql-general by date:

Previous
From: Stephen Frost
Date:
Subject: Re: Is there a way to speed up WAL replay?
Next
From: Nicolas Grilly
Date:
Subject: Re: Is there a way to speed up WAL replay?