Re: doc: virtual envs with Pl/Python - Mailing list pgsql-hackers

From Florents Tselai
Subject Re: doc: virtual envs with Pl/Python
Date
Msg-id CB5C263E-A887-4FB2-910F-F1EC7435D625@gmail.com
Whole thread Raw
In response to Re: doc: virtual envs with Pl/Python  (Peter Eisentraut <peter@eisentraut.org>)
Responses Re: doc: virtual envs with Pl/Python
List pgsql-hackers


On 17 Oct 2024, at 5:10 PM, Peter Eisentraut <peter@eisentraut.org> wrote:

On 17.10.24 15:56, Tom Lane wrote:
Florents Tselai <florents.tselai@gmail.com> writes:
I think there should be a mention of virtual environments in the plpython
docs.
Why?  We are not here to teach people how to use Python.  More, this
patch seems like not merely teaching Python, but teaching the use of
specific Python installation patterns that might or might not apply
on a particular platform.  That's not going to scale.

Virtual environments are a standard part of Python now, so maybe there is something to this.  But personally I don't understand this.  Do you mean for this to select the Python environment that PL/Python is built against, or do you want to use the virtual environment at run time?  The text says "installing" and talks about package managers.  It's unclear to me at exactly what step this is supposed to apply.


this 

want to use the virtual environment at run time? 


OK, I should have explained this a bit more. 

Here’s the story:

Unless someone builds from source, chances are, the default Python interpreter is something like /usr/bin/python3 .
Which sometimes may even point to python3.10/11/12 . 
Plus, If you have multiple Postgres versions (not unusual if you’ve had the same PG server for years),
the version matrix gets too complicated.

Which means that if you want non-standard packages in your Pl/Python code,
you’ll have to `sudo python3 -m pip install` things, which in turn will break things in different versions.
In fact, in the latest versions standard Python actively discourages that by having your
say `pip install … —break-system-packages` 

 
The practice now is to build a virtual `python3 -m venv`, which, as you say, is standard.
And indeed, it is not just a specific installation pattern. 
I’m not getting into fancy things like poetry et. al. here. 

The documentation as-is mentions the availability of env vars that can be set and stops there.

As-is, someone who reads the documentation is left with the impression that 
the only way to use non-standard Python libs is by `sudo python3 -m install package` because 
that’s the python3 interpreter that plpython3u picks-up during installation.

The proposed note goes a step further and aims to say how to support this not-so-rare pattern.

I hope that’s more clear now.

PS: fwiw, It’s practically impossible to build —with-python=/path/to/venv because most virtualenvs creators don’t bother handling header files correctly. config/python.m4 can’t support that. But that’s another story irrelevant here.

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: ECPG cleanup and fix for clang compile-time problem
Next
From: Jacob Champion
Date:
Subject: Re: doc: virtual envs with Pl/Python