The ultimate extension hook. - Mailing list pgsql-hackers

While I started just wanting the hook for the absolute minimum overhead to execute a function, even faster than fastpath, and in brainstorming with David Rowley other use cases became apparent.

API tracing within the engine.  I've heard of client tools for this.

API filtering.  Block/ignore manual checkpoints for instance.

API message altering.

Anything you want to hook into at the highest level well above ExecutorRun.

Originally I just wanted a lightweight mechanism to capture some system counters like /proc/stat without going through the SQL execution machinery.  I'm picky about implementing stuff in the absolute fastest way.  :-)  But I think there are other practical things that I haven't even thought of yet.

There are a few implementation mechanisms which achieve slightly different possibilities:

1) The generic mechanism would let one or more API filters be installed to directly call functions in an extension.  There would be no SQL arg processing overhead based on the specific function.   You'd just pass it the StringInfoData 'msg' itself. Multiple extensions might use the hook so you'd need to rewind the StringInfo buffer.  Maybe I return a boolean to indicate no further processing of this message or fall through to the normal "switch (firstchar)" processing.

2) switch (firstchar) { case 'A': // New trivial API message for extensions

which would call a single extension installed function to do whatever I wanted based on the message payload.  And, yes, I know this can be done just using SQL.  It is simply a variation.  But this would require client support and I prefer the below.

3) case 'Q':  /* simple query */

if (pq_peekbyte() == '!' && APIHook != NULL) {

   (*APIHook)(msg);

  <return something>

  ...

  continue;

}

I've use this last technique to do things like:

if (!strncmp(query_string, "DIEDIEDIE", 9) {

    char *np = NULL;

    *np = 1;

} else if (!strncmp(query_string, "PING", 4) {

    static const char *pong = "PONG";

    pq_putmessage('C', pong, strlen(pong) + 1);

    send_ready_for_query = true;

    continue;

} else if (...)

Then I can simple type PING into psql and get back a PONG.

Or during a stress test on a remote box I can execute the simple query "DIEDIEDIE" and crash the server.  I did this inline for experimentation before but it would be nice if I had the mechanism to use a "statement" to invoke a hook function in an extension.  A single check for "!" in the 'Q' processing would allow user defined commands in extensions.  The dispatcher would be in the extension.  I just need the "!" check.

Another example where ultimate performance might be a goal, if  you are familiar with why redis/memcached/etc. exists then imagine loading SQL results into a cache in an extension and executing as a 'simple' query something like:  !LOOKUP <KEY>

and getting the value faster than SQL could do.

Before I prototype I want to get some feedback.  Why not have a hook at the API level?