On Tue, Nov 17, 2020 at 10:21 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
2. It's difficult for extensions to use. There is a callback mechanism for extensions, but it's much less convenient to use than the built-in functions. The pgcrypto code uses the callbacks currently, and Michael's patch [2] would move that support for tracking OpenSSL contexts to the core, which makes it a lot more convenient for pgcrypto. Wouldn't it be nice if extensions could have the same ergonomics as built-in code, if they need to track external resources?
Yes, in general I'm a big fan of making life easier for extensions (see postscript), and this looks helpful. It's certainly fairly clear to read. I'm in-principle in favour, though I haven't read the old resowner code closely enough to have a strong opinion.
I haven't done thorough performance testing of this, but with some quick testing with Kyotaro's "catcachebench" to stress-test syscache lookups, this performs a little better. In that test, all the activity happens in the small array portion, but I believe that's true for most use cases.
Thoughts? Can anyone suggest test scenarios to verify the performance of this?
I didn't think of much really.
I have tossed together a patch on top of yours that adds some systemtap/dtrace tracepoints and provides a demo systemtap script that shows some basic stats collection done using them. My intention was to make it easier to see where the work in the resource owner code was being done. But after I did the initial version I realised that in this case it probably doesn't add a lot of value over using targeted profiling of only functions in resowner.c and their callees which is easy enough using perf and regular DWARF debuginfo. So I didn't land up polishing it and adding stats for the other counters. It's attached anyway, in case it's interesting or useful to anyone.
P.S. At some point I want to tackle some things similar to what you've done here for other kinds of backend resources. In particular, to allow extensions to register their own heavyweight lock types - with the ability to handle lock timeouts, and add callbacks in the deadlock detector to allow extensions to register dependency edges that are not natively visible to the deadlock detector and to choose victim priorities. Also some enhancements to how pg_depend tracking can be used by extensions. And I want to help get the proposed custom ProcSignal changes in too. I think there's a whole lot to be done to make extensions easier and safer to author in general, like providing a simple way to do error trapping and recovery in an extension mainloop without copy/pasting a bunch of PostgresMain code, better default signal handlers, startup/shutdown that shares more with user backends, etc. Right now it's quite tricky to get bgworkers to behave well. </wildhandwaving>
