Thread: How to manage shared library lifetime through C functions

Seref Arikan wrote:
> I hope this is the right group to ask this question; apologies if this should go the general or some
> other list.
> 
> 
> I have multiple shared libraries that can be called from C that I'd like to use from a C based
> postgresql function.
> 
> These libraries perform some expensive initialization and they require the C code to properly release
> resources when the library is no longer needed.
> 
> 
> This means that I need a mechanism to keep a session level pointer to a library, initialize it when it
> is called first from a C based function and dispose the library properly when the session ends (and
> terminated due to a problem) I would like to keep the libraries available as long as the session is
> alive, so multiple calls are supposed to avoid initialization/disposal costs every time.
> 
> 
> I could probably use a temp table as a container for the initalization and even pointer values (sounds
> dirty) but I have no idea how to hook to session end to clean up when session ends.
> 
> 
> What would be a good strategy here?

You could register a callback at process exit with on_proc_exit() from storage/ipc.h.

Yours,
Laurenz Albe

On 08/04/2014 12:54 PM, Seref Arikan wrote:
> Greetings,
> I hope this is the right group to ask this question; apologies if this
> should go the general or some other list.
>
> I have multiple shared libraries that can be called from C that I'd like to
> use from a C based postgresql function.
>
> These libraries perform some expensive initialization and they require the
> C code to properly release resources when the library is no longer needed.
>
> This means that I need a mechanism to keep a session level pointer to a
> library, initialize it when it is called first from a C based function and
> dispose the library properly when the session ends (and terminated due to a
> problem) I would like to keep the libraries available as long as the
> session is alive, so multiple calls are supposed to avoid
> initialization/disposal costs every time.
>
> I could probably use a temp table as a container for the initalization and
> even pointer values (sounds dirty) but I have no idea how to hook to
> session end to clean up when session ends.
>
> What would be a good strategy here?

Define a function called _PG_init() in your C extension. PostgreSQL will 
call it once, when the library is loaded into the backend. (The time 
it's loaded will depend on whether the library is listed in 
shared_preload_libraries, local_preload_libraries, or neither.)

Are you sure you need to do any cleanup? When the session ends, the 
backend process will terminate, which will close any open files and 
release memory that the library might be holding. If that's not enough, 
and the library really needs to do more complicated clean up, then you 
can register a callback with on_proc_exit().

Look at the C extensions in the PostgreSQL source tree's contrib 
directory for examples of _PG_init() and on_proc_exit().

- Heikki

On 08/04/2014 01:31 PM, Seref Arikan wrote:
> Thanks a lot Heikki and Albe. Exactly what I was asking for.
> Heikki: the libraries are written in languages that have their own runtime
> and their documentation insists that both init and dispose calls are
> performed when used from C. PG_init() and proc_exit sounds spot on.

You might also consider creating your own PL handler for those 
languages. Or google around if one exists already.

> Any ideas about keeping some data at session level between calls? Both
> calls of the same function and different C functions. (though temp table is
> always there as an option)

You can use a global variable in the C extension. If you need to 
allocate memory that survives across function calls, use 
"MemoryContextSwitchTo(TopMemoryContext); ... = palloc(...)". I'm sure 
you'll find examples of that in the existing extensions too.

- Heikki

On 08/04/2014 06:31 PM, Seref Arikan wrote:
> Thanks a lot Heikki and Albe. Exactly what I was asking for.
> Heikki: the libraries are written in languages that have their own
> runtime and their documentation insists that both init and dispose calls
> are performed when used from C. PG_init() and proc_exit sounds spot on.

That's usually with reference to code that might then load them again,
such as another library. Without specifics it's hard to say.

If these tools depend on the C code terminating cleanly to operate
correctly then they'll also break if the OS crashes, the system loses
power, etc.

Exiting a program without calling cleanup is in many ways just a
particularly friendly kind of crash, and it's often actually much more
efficient than doing cleanup. Why neatly tidy the floor when you're
about to demolish the house? Total waste of time.


-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

Craig Ringer wrote:
> On 08/04/2014 06:31 PM, Seref Arikan wrote:
>> Thanks a lot Heikki and Albe. Exactly what I was asking for.
>> Heikki: the libraries are written in languages that have their own
>> runtime and their documentation insists that both init and dispose calls
>> are performed when used from C. PG_init() and proc_exit sounds spot on.
> 
> That's usually with reference to code that might then load them again,
> such as another library. Without specifics it's hard to say.
> 
> If these tools depend on the C code terminating cleanly to operate
> correctly then they'll also break if the OS crashes, the system loses
> power, etc.
> 
> Exiting a program without calling cleanup is in many ways just a
> particularly friendly kind of crash, and it's often actually much more
> efficient than doing cleanup. Why neatly tidy the floor when you're
> about to demolish the house? Total waste of time.

There are valid use cases (else the function probably wouldn't exist).
I use it in oracle_fdw to gracefully close any open Oracle connections when
the process exits.

Yours,
Laurenz Albe

On 08/04/2014 09:48 PM, Albe Laurenz wrote:
> There are valid use cases (else the function probably wouldn't exist).
> I use it in oracle_fdw to gracefully close any open Oracle connections when
> the process exits.

True; it's sometimes better to do a clean exit.

It's relying on that always happening that's a problem. In your case
Oracle will clean up dead connections after a while, for example (or
quite promptly as the OS will rapidly TCP RST them). So it's just nicer.

Point taken though.


-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services