Support to define custom wait events for extensions - Mailing list pgsql-hackers

Hi,

Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to identify which
extension is the bottleneck.

So, I'd like to support new APIs to define custom wait events
for extensions. It's discussed in [1].

I made patches to realize it. Although I have some TODOs,
I want to know your feedbacks. Please feel free to comment.


# Implementation of new APIs

I implemented 2 new APIs for extensions.
* RequestNamedExtensionWaitEventTranche()
* GetNamedExtensionWaitEventTranche()

Extensions can request custom wait events by calling
RequestNamedExtensionWaitEventTranche(). After that, use
GetNamedExtensionWaitEventTranche() to get the wait event information.

The APIs usage example by extensions are following.

```
shmem_request_hook = shmem_request;
shmem_startup_hook = shmem_startup;

static void
shmem_request(void)
{
    /* request a custom wait event */
    RequestNamedExtensionWaitEventTranche("custom_wait_event");
}

static void
shmem_startup(void)
{
    /* get the wait event information */
    custom_wait_event = 
GetNamedExtensionWaitEventTranche("custom_wait_event");
}

void
extension_funtion()
{
    (void) WaitLatch(MyLatch,
                     WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
                     10L * 1000,
                     custom_wait_event);      /* notify core with custom wait event */
    ResetLatch(MyLatch);
}
```


# Implementation overview

I referenced the implementation of
RequestNamedLWLockTranche()/GetNamedLWLockTranche().
(0001-Support-to-define-custom-wait-events-for-extensions.patch)

Extensions calls RequestNamedExtensionWaitEventTranche() in
shmem_request_hook() to request wait events to be used by each 
extension.

In the core, the requested wait events are dynamically registered in 
shared
memory. The extension then obtains the wait event information with
GetNamedExtensionWaitEventTranche() and uses the value to notify the 
core
that it is waiting.

When a string representing of the wait event is requested,
it returns the name defined by calling 
RequestNamedExtensionWaitEventTranche().


# PoC extension

I created the PoC extension and you can use it, as shown here:
(0002-Add-a-extension-to-test-custom-wait-event.patch)

1. start PostgreSQL with the following configuration
shared_preload_libraries = 'inject_wait_event'

2. check wait events periodically
psql-1=# SELECT query, wait_event_type, wait_event FROM pg_stat_activity 
WHERE backend_type = 'client backend' AND pid != pg_backend_pid() ;
psql-1=# \watch

3. execute a function to inject a wait event
psql-2=# CREATE EXTENSION inject_wait_event;
psql-2=# SELECT inject_wait_event();

4. check the custom wait event
You can see the following results of psql-1.

(..snip..)

             query            | wait_event_type | wait_event
-----------------------------+-----------------+------------
  SELECT inject_wait_event(); | Extension       | Extension
(1 row)

(..snip..)

(...about 10 seconds later ..)


             query            | wait_event_type |    wait_event
-----------------------------+-----------------+-------------------
  SELECT inject_wait_event(); | Extension       | custom_wait_event       
  # requested wait event by the extension!
(1 row)

(..snip..)


# TODOs

* tests on windows (since I tested on Ubuntu 20.04 only)
* add custom wait events for existing contrib modules (ex. postgres_fdw)
* add regression code (but, it seems to be difficult)
* others? (Please let me know)


[1] 
https://www.postgresql.org/message-id/81290a48-b25c-22a5-31a6-3feff5864fe3%40gmail.com

Regards,

-- 
Masahiro Ikeda
NTT DATA CORPORATION