Re: parallelizing the archiver - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: parallelizing the archiver
Date
Msg-id 512BF1DA-48E5-4F05-B2E9-2286F59CB037@amazon.com
Whole thread Raw
In response to Re: parallelizing the archiver  (Andrey Borodin <x4mmm@yandex-team.ru>)
List pgsql-hackers
On 9/29/21, 9:49 PM, "Bossart, Nathan" <bossartn@amazon.com> wrote:
> I'm sure there are other ways to approach this, but I thought I'd give
> it a try to see what was possible and to get the conversation started.

BTW I am also considering the background worker approach that was
mentioned upthread.  My current thinking is that the backup extension
would define a special background worker that communicates with the
archiver via shared memory.  As noted upthread, this would enable
extension authors to do whatever batching, parallelism, etc. that they
want, and it should also prevent failures from taking down the
archiver process.  However, this approach might not make sense for
things like recovery_end_command that are only executed once.  Maybe
it's okay to leave that one alone for now.

Nathan


pgsql-hackers by date:

Previous
From: Jaime Casanova
Date:
Subject: Re: 2021-09 Commitfest
Next
From: Andrew Dunstan
Date:
Subject: Re: PATH manipulation in 001_libpq_pipeline.pl fails on windows