Thread: Re: pgsql: aio: Infrastructure for io_method=worker

Andres Freund <andres@anarazel.de> writes:
> To allow the number of IO workers to be increased without a restart, we need
> to reserve PGPROC entries for the workers unconditionally. This has been
> judged to be worth the cost. If it turns out to be problematic, we can
> introduce a PGC_POSTMASTER GUC to control the maximum number.

So I see this patch added 32 PGPROCs and hence 32 semaphores to the
system's requirements.  Unsurprisingly, this broke OpenBSD/NetBSD
again:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sawshark&dt=2025-03-18%2016%3A20%3A05

It's probably time to just abandon the idea of being able to run with
only 60 semaphores.  I wonder though if we ought to revert 38da05346
and/or 6d0154196 in view of that.

            regards, tom lane

Hi,

On 2025-03-18 18:53:48 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > To allow the number of IO workers to be increased without a restart, we need
> > to reserve PGPROC entries for the workers unconditionally. This has been
> > judged to be worth the cost. If it turns out to be problematic, we can
> > introduce a PGC_POSTMASTER GUC to control the maximum number.
> 
> So I see this patch added 32 PGPROCs and hence 32 semaphores to the
> system's requirements.

Yea - I brought that suspicion up a few times in the thread and in the
resulting discussion it wasn't deemed worth introducing a PGC_POSTMASTER guc
to control the max.


> It's probably time to just abandon the idea of being able to run with
> only 60 semaphores.

Agreed. It's an absurdly outdated OS default configuration. Realistically one
can't run postgres even in a toy scenario without changing it. So the value of
being able to test postgres with the default OS settings doesn't seem that
high. If it were on a very commonly used platform I would think differently,
but net/openbsd ain't that.


> I wonder though if we ought to revert 38da05346 and/or 6d0154196 in view of
> that.

38da05346 doesn't seem to have much value if it doesn't help us run the tests
by default - but it also doesn't really hurt. So, shrug, I guess.

6d0154196 - a higher autovacuum_worker_slots increases resource usage more
than IO workers do, because autovac workers are included in computations like
lock space. But 16 isn't that much either way...

Greetings,

Andres Freund

On Tue, Mar 18, 2025 at 07:07:53PM -0400, Andres Freund wrote:
> On 2025-03-18 18:53:48 -0400, Tom Lane wrote:
>> It's probably time to just abandon the idea of being able to run with
>> only 60 semaphores.
> 
> Agreed. It's an absurdly outdated OS default configuration. Realistically one
> can't run postgres even in a toy scenario without changing it. So the value of
> being able to test postgres with the default OS settings doesn't seem that
> high. If it were on a very commonly used platform I would think differently,
> but net/openbsd ain't that.

+1.  The platforms with low defaults do force us to think carefully about
increasing the default requirements for Postgres, but I don't think that's
a strong enough argument for the maintenance effort.

-- 
nathan

Andres Freund <andres@anarazel.de> writes:
> On 2025-03-18 18:53:48 -0400, Tom Lane wrote:
>> I wonder though if we ought to revert 38da05346 and/or 6d0154196 in view of
>> that.

> 38da05346 doesn't seem to have much value if it doesn't help us run the tests
> by default - but it also doesn't really hurt. So, shrug, I guess.

> 6d0154196 - a higher autovacuum_worker_slots increases resource usage more
> than IO workers do, because autovac workers are included in computations like
> lock space. But 16 isn't that much either way...

IMV the argument for reverting either would basically be simplicity.
It's not even so much the code, as the comments defending these odd
looking choices.  Future hackers will read those and wonder if the
arguments still apply --- and the answer will be "no".

38da05346 was back-patched into 17, and I'd leave it as-is there,
since it was holding the line on "can start with 60 semaphores"
for that branch.  But once we've lost that battle, it's hard to
see what it's doing for us.

            regards, tom lane