Thread: PL/Python - lifetime of variables?
Hello, I noticed that variables in PL/Python are not released at the end of procedure. Does it expected behavior? See this example below: https://github.com/heterodb/pg-strom/blob/master/test/input/arrow_python.source#L53 This PL/Python function maps a GPU buffer as cupy.ndarray object by cupy_strom.ipc_import() at the L59. It shall be stored in X. I have expected that X shall be released at end of the procedure invocation, but not happen. The object X internally hold IpcMemory class, https://github.com/heterodb/pg-strom/blob/master/python/cupy_strom.pyx And, it has destructor routine that unmaps the above GPU buffer using CUDA API. https://github.com/heterodb/pg-strom/blob/master/python/cupy_ipcmem.c#L242 Because of the restriction by CUDA API, we cannot map a certain GPU buffer twice on the same process space. So, I noticed that the second invocation of the PL/Python procedure on the same session failed. The L103 explicitly reset X, by X=0, to invoke the destructor manually. I wonder whether it is an expected behavior, or oversight of something. Best regards, -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>
On Wed, 19 Feb 2020 at 09:12, Kohei KaiGai <kaigai@heterodb.com> wrote: > I noticed that variables in PL/Python are not released at the end of procedure. > Does it expected behavior? PL/Python vars are freed when the interpreter instance is freed and their refcounts reach zero. I believe we use one subinterpreter for the lifetime of the backend session. It might be worth checking whether we do an eager refcount check and sweep when a procedure finishes. But in general, I suggest that relying on object finalizers/destructors to accomplish side effects visible outside the procedure is bad development practice. Instead, use a "with" block, or a try/finally block, and do explicit cleanup for external resources. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise