Re: O(n) tasks cause lengthy startups and checkpoints - Mailing list pgsql-hackers

From Robert Haas
Subject Re: O(n) tasks cause lengthy startups and checkpoints
Date
Msg-id CA+TgmoYkDY7r8PwAGhD3tMqVAeKBfY-_7miJKcj-ZzPWN0UYzQ@mail.gmail.com
Whole thread Raw
In response to Re: O(n) tasks cause lengthy startups and checkpoints  (Simon Riggs <simon.riggs@enterprisedb.com>)
Responses Re: O(n) tasks cause lengthy startups and checkpoints  (Simon Riggs <simon.riggs@enterprisedb.com>)
Re: O(n) tasks cause lengthy startups and checkpoints  (Nathan Bossart <nathandbossart@gmail.com>)
List pgsql-hackers
On Thu, Jun 23, 2022 at 7:58 AM Simon Riggs
<simon.riggs@enterprisedb.com> wrote:
> Having a central cleanup process makes a lot of sense. There is a long
> list of potential tasks for such a process. My understanding is that
> autovacuum already has an interface for handling additional workload
> types, which is how BRIN indexes are handled. Do we really need a new
> process?

It seems to me that if there's a long list of possible tasks for such
a process, that's actually a trickier situation than if there were
only one or two, because it may happen that when task X is really
urgent, the process is already busy with task Y.

I don't think that piggybacking more stuff onto autovacuum is a very
good idea for this exact reason. We already know that autovacuum
workers can get so busy that they can't keep up with the need to
vacuum and analyze tables. If we give them more things to do, that
figures to make it worse, at least on busy systems.

I do agree that a general mechanism for getting cleanup tasks done in
the background could be a useful thing to have, but I feel like it's
hard to see exactly how to make it work well. We can't just allow it
to spin up a million new processes, but at the same time, if it can't
guarantee that time-critical tasks get performed relatively quickly,
it's pretty worthless.

-- 
Robert Haas
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Aleksander Alekseev
Date:
Subject: Re: Pluggable toaster
Next
From: Robert Haas
Date:
Subject: Re: some aspects of our qsort might not be ideal