Re: O(n) tasks cause lengthy startups and checkpoints - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: O(n) tasks cause lengthy startups and checkpoints
Date
Msg-id 2B230EF6-7509-479C-8C5A-34B56A1B4063@amazon.com
Whole thread Raw
In response to Re: O(n) tasks cause lengthy startups and checkpoints  (Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>)
Responses Re: O(n) tasks cause lengthy startups and checkpoints
List pgsql-hackers
On 12/6/21, 3:44 AM, "Bharath Rupireddy" <bharath.rupireddyforpostgres@gmail.com> wrote:
> On Fri, Dec 3, 2021 at 11:50 PM Bossart, Nathan <bossartn@amazon.com> wrote:
>> I might hack something together for the separate worker approach, if
>> for no other reason than to make sure I really understand how these
>> functions work.  If/when a better idea emerges, we can alter course.
>
> Thanks. As I said upthread we've been discussing the approach of
> offloading some of the checkpoint tasks like (deleting snapshot files)
> internally for quite some time and I would like to share a patch that
> adds a new background cleaner process (currently able to delete the
> logical replication snapshot files, if required can be extended to do
> other tasks as well). I don't mind if it gets rejected. Please have a
> look.

Thanks for sharing!  I've also spent some time on a patch set, which I
intend to share once I have handling for all four tasks (so far I have
handling for CheckPointSnapBuild() and RemovePgTempFiles()).  I'll
take a look at your patch as well.

Nathan


pgsql-hackers by date:

Previous
From: "Bossart, Nathan"
Date:
Subject: Re: pg_replslotdata - a tool for displaying replication slot information
Next
From: "Bossart, Nathan"
Date:
Subject: Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?