Thread: pgbench with libevent?
Hello devs,
Pgbench is managing clients I/Os manually with select or poll. Much of
this could be managed by libevent.
Pros:
1. libevent is portable, stable, and widely used (eg Chromium, Memcached, PgBouncer).
2. libevent implements more I/O wait methods, which may be more efficient on some platforms
(eg FreeBSD kqueue, Windows wepoll in libevent 2.2 alpha), and hides portability issues.
3. it would remove significant portions of unattractive pgbench code, esp. in threadRun,
and around socket/poll abstraction and portability layer.
4. depending on the number of event loops, the client load could be shared more evenly.
currently a thread only manages its own clients, some client I/Os may be waiting to be
processed while other threads could be available to process them.
Cons:
1. it adds a libevent dependency to postgres. This may be a no go from the start.
2. this is a significant refactoring project which implies a new internal architecture and adds
new code to process and generate appropriate events.
3. libevent ability to function efficiently in a highly multithreaded environment
is unclear. Should there be one event queue which generate a shared work queue?
or several event queues, one per thread (which would remove the sharing pro)?
or something in between? Some experiments and configuratibility may be desirable.
This may also have an impact on pgbench user interface and output depending on the result,
eg there may be specialized event and worker threads, some statistics may be slightly
different, new options may be needed…
4. libevent development seems slugish, last bugfix was published 3 years ago, version
2.2 has been baking for years, but the development seems lively (+100 contributors).
Neutral?
1. BSD 3 clauses license.
Is pros > cons, or not? Other thoughts, pros, cons?
--
Fabien.
> Pgbench is managing clients I/Os manually with select or poll. Much of this > could be managed by libevent. Or maybe libuv (used by nodejs?). From preliminary testing libevent seems not too good at fine grain time management which are used for throttling, whereas libuv advertised that it is good at it, although what it does is yet to be seen. Note: libev had no updates in 8 years. -- Fabien.
On Mon, Aug 14, 2023 at 12:35 PM Fabien COELHO <coelho@cri.ensmp.fr> wrote: > > Pgbench is managing clients I/Os manually with select or poll. Much of this > > could be managed by libevent. > > Or maybe libuv (used by nodejs?). > > From preliminary testing libevent seems not too good at fine grain time > management which are used for throttling, whereas libuv advertised that it > is good at it, although what it does is yet to be seen. Do you think our WaitEventSet stuff could be good here, if made frontend-friendly?
> On Mon, Aug 14, 2023 at 12:35 PM Fabien COELHO <coelho@cri.ensmp.fr> wrote: >> > Pgbench is managing clients I/Os manually with select or poll. Much of this >> > could be managed by libevent. >> >> Or maybe libuv (used by nodejs?). >> >> From preliminary testing libevent seems not too good at fine grain time >> management which are used for throttling, whereas libuv advertised that it >> is good at it, although what it does is yet to be seen. > > Do you think our WaitEventSet stuff could be good here, if made > frontend-friendly? Interesting. In my understanding this also needs to make Latch frontend-friendly? Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp
On Mon, Aug 14, 2023 at 6:07 PM Tatsuo Ishii <ishii@sraoss.co.jp> wrote: > Interesting. In my understanding this also needs to make Latch > frontend-friendly? It could be refactored to support a different subset of event types -- maybe just sockets, no latches and obviously no 'postmaster death'. But figuring out how to make latches work between threads might also be interesting for future projects... Maybe Fabien has completion-based I/O in mind (not just "readiness"). That's something that some of those libraries can do, IIUC. For example, when your thread wakes up, it tells you "your socket read is finished, the data is already in your target buffer". As opposed to "you can now call recv() without blocking", so you avoid another trip into the kernel. But that's also something we'll eventually want to figure out in the server.
On 2023-Aug-13, Fabien COELHO wrote: > 4. libevent development seems slugish, last bugfix was published 3 years ago, version > 2.2 has been baking for years, but the development seems lively (+100 contributors). Ugh, I would stay away from something like that. Would we become hostage to an undelivering group? No thanks. On 2023-Aug-14, Fabien COELHO wrote: > Or maybe libuv (used by nodejs?). > Note: libev had no updates in 8 years. libev or libuv? No updates in 8 years => dead. No way. Reworking based on wait events as proposed downthread sounds more promising. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual)
Hello Thomas, >>> Pgbench is managing clients I/Os manually with select or poll. Much of this >>> could be managed by libevent. >> >> Or maybe libuv (used by nodejs?). >> >> From preliminary testing libevent seems not too good at fine grain time >> management which are used for throttling, whereas libuv advertised that it >> is good at it, although what it does is yet to be seen. > > Do you think our WaitEventSet stuff could be good here, if made > frontend-friendly? Interesting question. AFAICS, the answer is that it could indeed probably fit the task, but it would require significant work to make it thread-compatible, and to untangle it from IsUnderPosmaster/postmaster death, memory context, elog/ereport, and other back-end specific stuff. If you remove all that with a clean abstraction (quite a task), then once done the question could be why not use libevent/libuv/… in the backend instead of maintaining more or less the same thing inside postgres? So ISTM that as far as pgbench is concerned it would be much simpler to use libevent/libuv/… directly if the pros are enough and the cons not redhibitory, and provided that the needed detailed features are really there. -- Fabien.
>> Interesting. In my understanding this also needs to make Latch >> frontend-friendly? > > It could be refactored to support a different subset of event types -- > maybe just sockets, no latches and obviously no 'postmaster death'. > But figuring out how to make latches work between threads might also > be interesting for future projects... > > Maybe Fabien has completion-based I/O in mind (not just "readiness"). Pgbench is really a primitive client on top of libpq. ISTM that completion-based I/O would require to enhance libpq asynchronous-ity, not just expose its underlying fd to allow asynchronous implementations. Currently pgbench only actuall "waits" for results from the server and testing PQisBusy to check whether they are there. > That's something that some of those libraries can do, IIUC. For > example, when your thread wakes up, it tells you "your socket read is > finished, the data is already in your target buffer". Indeed, libevent has a higher level "buffer" oriented API. > As opposed to "you can now call recv() without blocking", so you avoid > another trip into the kernel. But that's also something we'll > eventually want to figure out in the server. -- Fabien.
>> 4. libevent development seems slugish, last bugfix was published 3 years ago, version >> 2.2 has been baking for years, but the development seems lively (+100 contributors). > > Ugh, I would stay away from something like that. Would we become > hostage to an undelivering group? No thanks. Ok. >> Or maybe libuv (used by nodejs?). > >> Note: libev had no updates in 8 years. > > libev or libuv? No updates in 8 years => dead. No way. Sorry, it was not a typo, but the information was not very explicit. I have looked at 3 libraries: libevent, libuv and libev. libuv is quite lively, last updated 2023-06-30. libev is an often cited library, which indeed seems quite dead, so I was "noting" that I had discarded it, but it looked like a typo. > Reworking based on wait events as proposed downthread sounds more > promising. The wait event postgres backend implementation would require a lot of work to be usable in a client context. My current investigation is that libuv could be the reasonable target, if any, especially as it seems to provide a portable thread pool as well. -- Fabien.
> It could be refactored to support a different subset of event types -- > maybe just sockets, no latches and obviously no 'postmaster death'. Ok. > But figuring out how to make latches work between threads might also > be interesting for future projects... Maybe. Some people are working on threading PostgreSQL. They may already know... > Maybe Fabien has completion-based I/O in mind (not just "readiness"). > That's something that some of those libraries can do, IIUC. For > example, when your thread wakes up, it tells you "your socket read is > finished, the data is already in your target buffer". As opposed to > "you can now call recv() without blocking", so you avoid another trip > into the kernel. But that's also something we'll eventually want to > figure out in the server. Agreed. -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp