Re: Weird failure with latches in curculio on v15 - Mailing list pgsql-hackers

From Nathan Bossart
Subject Re: Weird failure with latches in curculio on v15
Date
Msg-id 20230205235747.GA275913@nathanxps13
Whole thread Raw
In response to Re: Weird failure with latches in curculio on v15  (Andres Freund <andres@anarazel.de>)
Responses Re: Weird failure with latches in curculio on v15
List pgsql-hackers
On Sun, Feb 05, 2023 at 03:01:57PM -0800, Andres Freund wrote:
> I think at the very least you'd want to have a separate callback for
> restoring segments than for restoring other files. But more likely a
> separate callback for each type of file to be restored.
> 
> For the timeline history case an parameter indicating that we don't want
> to restore the file, just to see if there's a conflict, would make
> sense.

That seems reasonable.

> For the segment files, we'd likely need a parameter to indicate whether
> the restore is random or not.

Wouldn't this approach still require each module to handle restoring ahead
of time?  I agree that the shell overhead isn't the main performance issue,
but it's unclear to me how much of this should be baked into PostgreSQL.  I
mean, we could introduce a GUC that tells us how far ahead to restore and
have a background worker (or multiple background workers) asynchronously
pull files into a staging directory via the callbacks.  Is that the sort of
scope you are envisioning?

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)
Next
From: Andres Freund
Date:
Subject: Re: Weird failure with latches in curculio on v15