On 29/07/2024 19:56, Robert Haas wrote:
> On Wed, Jul 24, 2024 at 8:58 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> a) assign every child process a PGPROC entry, and make postmaster
>> responsible for assigning them like you suggest. We'll need more PGPROC
>> entries, because currently a process doesn't reserve one until
>> authentication has happened. Or we change that behavior.
>
> I wonder how this works right now. Is there something that limits the
> number of authentication requests that can be in flight concurrently,
> or is it completely uncapped (except by machine resources)?
> My first thought when I read this was that it would be bad to have to
> put a limit on something that's currently unlimited. But then I
> started thinking that, even if it is currently unlimited, that might
> be a bug rather than a feature. If you have hundreds of pending
> authentication requests, that just means you're using a lot of machine
> resources on something that doesn't really help anybody. A machine
> with hundreds of authentication-pending connections is possibly
> getting DDOS'd and probably getting buried. You'd be better off
> focusing the machine's limited resources on the already-established
> connections and a more limited number of new connection attempts. If
> you accept so many connection attempts that you don't actually have
> enough memory/CPU/kernel scheduling firepower to complete the
> authentication process with any of them, it does nobody any good.
>
> I'm not sure what's best to do here; just thinking out loud.
Yes, there's a limit, roughly 2x max_connections. see
MaxLivePostmasterChildren().
There's another issue with that that I was about to post in another
thread, but here goes: the MaxLivePostmasterChildren() limit is shared
by all regular backends, bgworkers and autovacuum workers. If you open a
lot of TCP connections to postmaster and don't send anything to the
server, you exhaust those slots, and the server won't be able to start
any autovacuum workers or background workers either. That's not great. I
started to work on approach b), with separate pools of slots for
different kinds of child processes, which fixes that. Stay tuned for a
patch.
In addition to that, you can have an unlimited number of "dead-end"
backends, which are doomed to just respond with "sorry, too many
clients" error. The only limit on those is the amount of resources
needed for all the processes and a little memory to track them.
--
Heikki Linnakangas
Neon (https://neon.tech)
