Re: recovery modules - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: recovery modules
Date
Msg-id Y9RrJraFnMr3xWqt@paquier.xyz
Whole thread Raw
In response to Re: recovery modules  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Fri, Jan 27, 2023 at 04:09:39PM -0800, Andres Freund wrote:
> I think it would be hard to write a good module that isn't just implementing
> the existing commands without it. Needing to clean up archives and reacting to
> the end of recovery seems a pretty core task.

FWIW, recovery_end_command is straight-forward as it is done by the
startup process, so that's an easy take.  You could split the cake
into two parts, as well, aka first focus on restore_command and
recovery_end_command as a first step, then we could try to figure out
how archive_cleanup_command would fit in this picture with the
checkpointer or a different process.  There are a bunch of deployments
where WAL archive retention is controlled by the age of the backups,
not by the backend deciding when these should be removed as a
checkpoint runs depending on the computed redo LSN, so recovery
modules would still be useful with just coverage for restore_command
and recovery_end_command.

> I was briefly wondering whether it'd be worth trying to offload things like
> archive_cleanup_command from checkpointer to a different process, for
> robustness. But given that it's pretty much required for performance that the
> module runs in the startup process, that ship probably has sailed.

Yeah, agreed that this could be interesting.  That could leverage the
work of the checkpointer.  Nathan has proposed a patch for that
recently, as far as I recall, to offload some tasks from the startup
and checkpointer processes:
https://commitfest.postgresql.org/41/3448/

So that pretty much goes into the same area?
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: recovery modules
Next
From: Tomas Vondra
Date:
Subject: Re: Syncrep and improving latency due to WAL throttling