pgbench with libevent? - Mailing list pgsql-hackers

From Fabien COELHO
Subject pgbench with libevent?
Date
Msg-id f6a5e724-e2d2-af78-4a6-4547994761b1@minesparis.psl.eu
Whole thread Raw
Responses Re: pgbench with libevent?
Re: pgbench with libevent?
List pgsql-hackers
Hello devs,

Pgbench is managing clients I/Os manually with select or poll. Much of 
this could be managed by libevent.

Pros:

1. libevent is portable, stable, and widely used (eg Chromium, Memcached, PgBouncer).

2. libevent implements more I/O wait methods, which may be more efficient on some platforms
    (eg FreeBSD kqueue, Windows wepoll in libevent 2.2 alpha), and hides portability issues.

3. it would remove significant portions of unattractive pgbench code, esp. in threadRun,
    and around socket/poll abstraction and portability layer.

4. depending on the number of event loops, the client load could be shared more evenly.
    currently a thread only manages its own clients, some client I/Os may be waiting to be
    processed while other threads could be available to process them.

Cons:

1. it adds a libevent dependency to postgres. This may be a no go from the start.

2. this is a significant refactoring project which implies a new internal architecture and adds
    new code to process and generate appropriate events.

3. libevent ability to function efficiently in a highly multithreaded environment
    is unclear. Should there be one event queue which generate a shared work queue?
    or several event queues, one per thread (which would remove the sharing pro)?
    or something in between? Some experiments and configuratibility may be desirable.
    This may also have an impact on pgbench user interface and output depending on the result,
    eg there may be specialized event and worker threads, some statistics may be slightly
    different, new options may be needed…

4. libevent development seems slugish, last bugfix was published 3 years ago, version
    2.2 has been baking for years, but the development seems lively (+100 contributors).

Neutral?

1. BSD 3 clauses license.

Is pros > cons, or not? Other thoughts, pros, cons?

-- 
Fabien.

pgsql-hackers by date:

Previous
From: Fabien COELHO
Date:
Subject: Re: pgbench: allow to exit immediately when any client is aborted
Next
From: Andrew Dunstan
Date:
Subject: Re: run pgindent on a regular basis / scripted manner