Thread: pgsql: Prepare for Python "Limited API" in PL/Python

Prepare for Python "Limited API" in PL/Python

Using the Python Limited API would allow building PL/Python against
any Python 3.x version and using another Python 3.x version at run
time.  This commit does not activate that, but it prepares the code to
only use APIs supported by the Limited API.

Implementation details:

- Convert static types to heap types
  (https://docs.python.org/3/howto/isolating-extensions.html#heap-types).

- Replace PyRun_String() with component functions.

- Replace PyList_SET_ITEM() with PyList_SetItem().

Reviewed-by: Jakob Egger <jakob@eggerapps.at>
Discussion: https://www.postgresql.org/message-id/flat/ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/c47e8df815c1c45f4e4fc90d5817d67ab088279f

Modified Files
--------------
src/pl/plpython/plpy_cursorobject.c  | 71 +++++++++++++++++---------
src/pl/plpython/plpy_planobject.c    | 61 ++++++++++++++--------
src/pl/plpython/plpy_procedure.c     |  5 +-
src/pl/plpython/plpy_resultobject.c  | 98 ++++++++++++++++++++++--------------
src/pl/plpython/plpy_subxactobject.c | 41 ++++++++-------
src/pl/plpython/plpy_typeio.c        |  6 +--
6 files changed, 177 insertions(+), 105 deletions(-)

Peter Eisentraut <peter@eisentraut.org> writes:
> Prepare for Python "Limited API" in PL/Python

Buildfarm's not happy with this at all.  It fails here too,
with python 3.6.8:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  PyDict_GetItem (op=<unknown at remote 0x1>, key='enter')
    at /usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/dictobject.c:1411
1411        if (!PyDict_Check(op))
(gdb) bt
#0  PyDict_GetItem (op=<unknown at remote 0x1>, key='enter') at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/dictobject.c:1411
#1  0x00007fd48af71196 in _PyType_Lookup (name='enter', type=0x2236858) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/typeobject.c:2966
#2  _PyType_Lookup (name='enter', type=0x2236858) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/typeobject.c:2913
#3  _PyObject_GenericGetAttrWithDict (dict=0x0, name='enter', obj=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/object.c:1090
#4  PyObject_GenericGetAttr (name='enter', obj=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/object.c:1158
#5  PyObject_GetAttr (v=<optimized out>, name='enter') at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Objects/object.c:928
#6  0x00007fd48afc52f2 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:2873
#7  0x00007fd48afa1108 in PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7fd48a5d6588, for file <string>, line 4, in
__plpython_procedure_subtransaction_alternative_names_49201(s=<PLySubtransaction at remote 0x7fd497f321c8>)) 
    at /usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:752
#8  _PyFunction_FastCall (globals=<optimized out>, nargs=<optimized out>, args=<optimized out>, co=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4934
#9  fast_function (func=<optimized out>, stack=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4969
#10 0x00007fd48afc4537 in call_function (pp_stack=0x7ffe3d639da8, oparg=<optimized out>, kwnames=0x0) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4873
#11 0x00007fd48afc5198 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:3336
#12 0x00007fd48af210a4 in PyEval_EvalFrameEx (throwflag=0, f=Frame 0x7fd48a5b0048, for file <string>, line 1, in
<module>()) at /usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4167 
#13 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized
out>,argcount=<optimized out>, kwnames=<optimized out>, kwargs=<optimized out>, kwcount=<optimized out>,  
    kwstep=<optimized out>, defs=<optimized out>, defcount=<optimized out>, kwdefs=<optimized out>, closure=<optimized
out>,name=<optimized out>, qualname=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4167
#14 0x00007fd48af22463 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0,
argcount=0,args=0x0, locals=<optimized out>, globals=<optimized out>, _co=<optimized out>) 
    at /usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:4188
#15 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at
/usr/src/debug/python3-3.6.8-69.el8_10.x86_64/Python/ceval.c:732
#16 0x00007fd48b3773d7 in PLy_procedure_call (kargs=kargs@entry=0x7fd48b37eeda "args", vargs=<optimized out>,
proc=<optimizedout>, proc=<optimized out>) at plpy_exec.c:1069 
#17 0x00007fd48b377816 in PLy_exec_function (fcinfo=fcinfo@entry=0x226f948, proc=0x2269130) at plpy_exec.c:102
#18 0x00007fd48b3782c8 in plpython3_call_handler (fcinfo=0x226f948) at plpy_main.c:244
#19 0x00000000006eec15 in ExecInterpExpr (state=0x226f7f0, econtext=0x226f498, isnull=<optimized out>) at
execExprInterp.c:912
#20 0x0000000000727e80 in ExecEvalExprSwitchContext (isNull=0x7ffe3d63a3df, econtext=0x226f498, state=0x226f7f0) at
../../../src/include/executor/executor.h:397
#21 ExecProject (projInfo=0x226f7e8) at ../../../src/include/executor/executor.h:431
#22 ExecResult (pstate=<optimized out>) at nodeResult.c:135


            regards, tom lane

On 26.02.25 17:51, Tom Lane wrote:
> Peter Eisentraut <peter@eisentraut.org> writes:
>> Prepare for Python "Limited API" in PL/Python
> 
> Buildfarm's not happy with this at all.  It fails here too,
> with python 3.6.8:

Yeah, apparently, it needs some more analysis for these older versions. 
I have reverted it for now.