Proposal for async support in libpq - Mailing list pgsql-hackers

Here's what I propose to do with libpq to make it more useful for
real-time applications.  Any comments or better ideas?

The point of these changes is first to allow frontend applications to
receive NOTIFY responses without having to generate dummy queries,
and second to allow a frontend to perform other work while awaiting
the result of a query.

We can't break existing code for this, so the behavior of PQexec()
can't change.  Instead, I propose new functions to add to the API.
Internally, PQexec will be reimplemented in terms of these new
functions, but old apps won't notice any difference.

The new functions are:

    int PQexecAsync (PGconn *conn, const char *query);

Submits a query without waiting for the result.  Returns TRUE if the
query has been successfully dispatched, otherwise FALSE (in the FALSE
case, an error message is left in conn->errorMessage).

    typedef enum {
        PGASYNC_IDLE,
        PGASYNC_BUSY,
        PGASYNC_DONE
    } PGAsyncStatusType;

    PGAsyncStatusType PQasyncStatus (PGconn *conn);

Indicates the current status of an asynchronous query:
    PGASYNC_IDLE: nothing doing
    PGASYNC_BUSY: async query in progress
    PGASYNC_DONE: query done, can retrieve result with PQasyncResult
When the state is PGASYNC_DONE, calling PQasyncResult will reset the state
to PGASYNC_IDLE.  A new query can only be submitted in the IDLE state.

    PGresult* PQasyncResult (PGconn *conn);

If the state is PGASYNC_DONE and the query was successful, a PGresult
block is returned (which the caller must eventually free).  In all other
cases, NULL is returned and a suitable error message is left in
conn->errorMessage.  Also, if the state is PGASYNC_DONE then it is
reset to PGASYNC_IDLE.

    void PQconsumeInput (PGconn *conn);

This can be called at any time to check for and process new input from
the backend.  It returns no status indication, but after calling it
the application can inspect PQasyncStatus() and/or PQnotifies()
to see if a query was completed or a NOTIFY message arrived.

    int PQsocket (PGconn *conn);

Returns the Unix file descriptor for the socket connection to the
backend, or -1 if there is no open connection.  This is a violation of
modularity, of course, but there is no alternative: an application using
this facility needs to be able to use select() to wait for input from
either the backend or any other input streams it may have.  To use
select() the underlying socket must be made visible.

    PGnotify *PQnotifies (PGconn *conn);

This function doesn't need to change; we just observe that notifications
may become available as a side effect of executing either PQexec() or
PQconsumeInput().


The general assumption is that the application's main loop will use
select() to wait for input.  If select() indicates that input is
pending from the backend, then the app will call PQconsumeInput,
followed by checking PQasyncStatus() and/or PQnotifies().

I expect a lot of people would build "partially async" applications that
still do all the queries through PQexec(), but detect notifies
asynchronously via select/PQconsumeInput/PQnotifies.  This compromise
would allow notifies to be detected without issuing null queries,
without complicating the basic logic of issuing a series of queries.

The same functionality should be added to libpq++.


Some issues to be resolved:

1. The above API assumes that only one query can be outstanding at a
time (per connection).  Is there any prospect that the backends will
ever be able to handle multiple concurrent queries?  If so, we should
design the API so that PQexecAsync returns some kind of "active query"
object that's separate from the connection object.  Then PQasyncStatus
and PQasyncResult would apply to these objects individually (probably
they should act a little differently than given above, too).

2. Any comments about the naming conventions I used?  The existing code
seems a tad inconsistent; what is considered the right practice as to
capitalization etc?

            regards, tom lane