plpython is broken for recursive use - Mailing list pgsql-hackers

From Tom Lane
Subject plpython is broken for recursive use
Date
Msg-id 12322.1444939834@sss.pgh.pa.us
Whole thread Raw
List pgsql-hackers
I looked into bug #13683,
http://www.postgresql.org/message-id/20151015135804.3019.31908@wrigleys.postgresql.org

It looks to me like plpython basically doesn't work at all for re-entrant
calls to plpython functions, because all executions of a given function
share the same "globals" dict, which doesn't seem sane.  Shouldn't each
execution have its own?

I believe the specific sequence of events shown here is

1. On the way into the recursion, each level does PLy_function_build_args
which sets "i" as a key in the proc's globals dict.

2. On the way out, each level does PLy_function_delete_args which deletes
"i" from the dict.  At levels after the innermost, this results in setting
"KeyError: 'i'" as the active Python error, because that key is already
gone.  We fail to notice the error indicator, though.

3. The pending error indicator causes the attempt to "def" the second
function to fail.

If I change the test case to

create or replace function a(i integer) returns integer as
$$
if i > 1:   return i
j = plpy.execute("select a(%s)" % (i+1))[0]['a']
return i+j
$$ language plpythonu;

select a(0);

then I get

ERROR:  spiexceptions.ExternalRoutineException: NameError: global name 'i' is not defined
CONTEXT:  Traceback (most recent call last): PL/Python function "a", line 4, in <module>   j = plpy.execute("select
a(%s)"% (i+1))[0]['a']
 
PL/Python function "a"

which shows that a()'s own access to "i" is broken too, though I confess
I'm not sure why it's failing at line 4 rather than 5.

I think this means that we should get rid of proc->globals and instead
manufacture a new globals dict locally in each call to PLy_exec_function
or PLy_exec_trigger.  For SETOF functions it would be necessary to keep
the globals dict reference somewhere in the FunctionCallInfo struct,
probably.  Not sure about cleaning up after an error that occurs between
SETOF callbacks --- we might need plpython to grow an at-abort callback to
do decref's on unreleased dicts.

Thoughts?
        regards, tom lane



pgsql-hackers by date:

Previous
From: Claudio Freire
Date:
Subject: Re: Duda
Next
From: Josh Berkus
Date:
Subject: Re: plpython is broken for recursive use