Re: WIP: WAL prefetch (another approach) - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: WIP: WAL prefetch (another approach)
Date
Msg-id 20201113151344.GU16415@tamriel.snowman.net
Whole thread Raw
In response to Re: WIP: WAL prefetch (another approach)  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
Responses Re: WIP: WAL prefetch (another approach)  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers
Greetings,

* Tomas Vondra (tomas.vondra@enterprisedb.com) wrote:
> On 11/13/20 3:20 AM, Thomas Munro wrote:
> > I'm not really sure what to do about achive restore scripts that
> > block.  That seems to be fundamentally incompatible with what I'm
> > doing here.
>
> IMHO we can't do much about that, except for documenting it - if the
> prefetch can't work because of blocking restore script, someone has to
> fix/improve the script. No way around that, I'm afraid.

I'm a bit confused about what the issue here is- is the concern that a
restore_command is specified that isn't allowed to run concurrently but
this patch is intending to run more than one concurrently..?  There's
another patch that I was looking at for doing pre-fetching of WAL
segments, so if this is also doing that we should figure out which
patch we want..

I don't know that it's needed, but it feels likely that we could provide
a better result if we consider making changes to the restore_command API
(eg: have a way to say "please fetch this many segments ahead, and you
can put them in this directory with these filenames" or something).  I
would think we'd be able to continue supporting the existing API and
accept that it might not be as performant.

Thanks,

Stephen

Attachment

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: [PATCH] remove deprecated v8.2 containment operators
Next
From: Tom Lane
Date:
Subject: Re: [PATCH] remove deprecated v8.2 containment operators