On Sat, Aug 31, 2024 at 10:17 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> > * This direction seems to fit quite nicely with future ideas about
> > asynchronous network I/O. That may sound unrelated, but imagine that
> > a future version of WaitEventSet is built on Linux io_uring (or
> > Windows iorings, or Windows IOCP, or kqueue), and waits for the kernel
> > to tell you that network data has been transferred directly into a
> > user space buffer. You could wait for the interrupt word to change at
> > the same time by treating it as a futex[1]. Then all that other stuff
> > -- signalfd, is_set, maybe_sleeping -- just goes away, and all we have
> > left is one single word in memory. (That it is possible to do that is
> > not really a coincidence, as our own Mr Freund asked Mr Axboe to add
> > it[2]. The existing latch implementation techniques could be used as
> > fallbacks, but when looked at from the right angle, once you squish
> > all the wakeup reasons into a single word, it's all just an
> > implementation of a multiplexable futex with extra steps.)
>
> Cool
Just by the way, speaking of future tricks and the connections between
this code and other problems in other threads, I wanted to mention
that the above thought is also connected to CF #3998. When I started
working on this, in parallel I had an experimental patch set using
futexes[1] (back then, to try out futexes, I had to patch my OS[2]
because Linux couldn't multiplex them yet, and macOS/*BSD had
something sort of vaguely similar but effectively only usable between
threads in one process). I planned to switch to waiting directly on
the interrupt vector as a futex when bringing that idea together with
the one in this thread, but I guess I assumed we had to keep latches
too since they seemed like such a central concept in PostgreSQL. Your
idea seems much better, the more I think about it, but maybe only the
inventor of latches could have the idea of blowing them up :-)
Anyway, in that same experiment I realised I could wake multiple
backends in one system call, which led to more discoveries about the
negative interactions between latches and locks, and begat CF #3998
(SetLatches()). By way of excuse, unfortunately I got blocked in my
progress on interrupt vectors for a couple of release cycles by the
recovery conflict system, a set of procsignals that were not like the
others, and turned out to be broken more or less as a result. That
was tricky to fix (CF #3615), leading to journeys into all kinds of
strange places like the regex code...
[1] https://github.com/macdice/postgres/commits/kqueue-usermem/
[2] https://reviews.freebsd.org/D37102
