Re: parallelizing the archiver - Mailing list pgsql-hackers

From Julien Rouhaud
Subject Re: parallelizing the archiver
Date
Msg-id CAOBaU_a_xvFRKvEmQ-pnv1x-mrDHmAk6oWje+GsF8WDpK1Qhiw@mail.gmail.com
Whole thread Raw
In response to Re: parallelizing the archiver  (Andrey Borodin <x4mmm@yandex-team.ru>)
Responses Re: parallelizing the archiver  (Andrey Borodin <x4mmm@yandex-team.ru>)
Re: parallelizing the archiver  (Stephen Frost <sfrost@snowman.net>)
List pgsql-hackers
On Fri, Sep 10, 2021 at 2:03 PM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>
> > 10 сент. 2021 г., в 10:52, Julien Rouhaud <rjuju123@gmail.com> написал(а):
> >
> > Yes, but it also means that it's up to every single archiving tool to
> > implement a somewhat hackish parallel version of an archive_command,
> > hoping that core won't break it.
> I'm not proposing to remove existing archive_command. Just deprecate it one-WAL-per-call form.

Which is a big API beak.

> It's a very simplistic approach. If some GUC is set - archiver will just feed ready files to stdin of archive
command.What fundamental design changes we need? 

I'm talking about the commands themselves.  Your suggestion is to
change archive_command to be able to spawn a daemon, and it looks like
a totally different approach.  I'm not saying that having a daemon
based approach to take care of archiving is a bad idea, I'm saying
that trying to fit that with the current archive_command + some new
GUC looks like a bad idea.



pgsql-hackers by date:

Previous
From: Andrey Borodin
Date:
Subject: Re: parallelizing the archiver
Next
From: Andrey Borodin
Date:
Subject: Re: parallelizing the archiver