Re: parallelizing the archiver - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: parallelizing the archiver
Date
Msg-id BE033CAA-6541-4565-AF5D-4FA18F7EFF68@amazon.com
Whole thread Raw
In response to Re: parallelizing the archiver  (Julien Rouhaud <rjuju123@gmail.com>)
Responses Re: parallelizing the archiver  (Julien Rouhaud <rjuju123@gmail.com>)
List pgsql-hackers
On 9/7/21, 11:38 PM, "Julien Rouhaud" <rjuju123@gmail.com> wrote:
> On Wed, Sep 8, 2021 at 6:36 AM Bossart, Nathan <bossartn@amazon.com> wrote:
>>
>> I'd like to gauge interest in parallelizing the archiver process.
>> [...]
>> Based on previous threads I've seen, I believe many in the community
>> would like to replace archive_command entirely, but what I'm proposing
>> here would build on the existing tools.
>
> Having a new implementation that would remove the archive_command is
> probably a better long term solution, but I don't know of anyone
> working on that and it's probably gonna take some time.  Right now we
> have a lot of users that face archiving bottleneck so I think it would
> be a good thing to implement parallel archiving, fully compatible with
> current archive_command, as a short term solution.

Thanks for chiming in.  I'm planning to work on a patch next week.

Nathan


pgsql-hackers by date:

Previous
From: "Bossart, Nathan"
Date:
Subject: Re: .ready and .done files considered harmful
Next
From: "Bossart, Nathan"
Date:
Subject: Re: pg_walinspect - a new extension to get raw WAL data and WAL stats