Re: Weird failure with latches in curculio on v15 - Mailing list pgsql-hackers

From Nathan Bossart
Subject Re: Weird failure with latches in curculio on v15
Date
Msg-id 20230209002413.GA603595@nathanxps13
Whole thread Raw
In response to Re: Weird failure with latches in curculio on v15  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Weird failure with latches in curculio on v15
List pgsql-hackers
On Thu, Feb 09, 2023 at 08:56:24AM +0900, Michael Paquier wrote:
> On Wed, Feb 08, 2023 at 02:25:54PM -0800, Nathan Bossart wrote:
>> These are all good points.  Perhaps there could be a base archiver
>> implementation that shell_archive uses (and that other modules could use if
>> desired, which might be important for backward compatibility with the
>> existing callbacks).  But if you want to do something fancier than
>> archiving sequentially, you could write your own.
> 
> Which is basically the kind of things you can already achieve with a
> background worker and a module of your own?

IMO one of the big pieces that's missing is a way to get the next N files
to archive.  Right now, you'd have to trawl through archive_status on your
own if you wanted to batch/parallelize.  I think one advantage of what
Robert is suggesting is that we could easily provide a supported way to get
the next set of files to archive, and we can asynchronously mark them
"done".  Otherwise, each module has to implement this.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: OpenSSL 3.0.0 vs old branches
Next
From: Andres Freund
Date:
Subject: Re: tests against running server occasionally fail, postgres_fdw & tenk1