Thread: Switching PL/Python to Python 3 by default in PostgreSQL 12
Greetings:
I am not sure if this has been brought up before but Python 2 is EOL on Jan 1 2020. After that time there will not be any security fixes or patches.
According to our most recent official documentation:
https://www.postgresql.org/docs/11/plpython-python23.html
https://www.postgresql.org/docs/11/plpython-python23.html
" The default will probably be changed to Python 3 in a distant future release of PostgreSQL, depending on the progress of the migration to Python 3 in the Python community."
I know we are late in the Postgresql 12 cycle but I think switching the default to Python 3 is warranted given:
1. The serious nature of not having a default supported Python version soon after the PostgreSQL 12 release
2. The next opportunity to change the default will be late 2020
If we do not switch our default version and a vulnerability arises in Python 2 then we will end up either
1. Telling our users to run the default PL/Python with a known security vulnerability
2. The PostgreSQL community patching it's python
I know there are implications for swapping the default version but I think that is outweighed by the seriousness of the situation.
Thanks
Steve
On 2019-07-06 21:02, Steven Pousty wrote: > /" The default will probably be changed to Python 3 in a distant future > release of PostgreSQL, depending on the progress of the migration to > Python 3 in the Python community."/ > > I know we are late in the Postgresql 12 cycle but I think switching the > default to Python 3 is warranted given: "The default" in the above statement refers to what version the language name "plpythonu" invokes. That is separate from whether an installer or packager chooses to install a Python-2-based version. I expect that as operating systems remove Python 2 from their systems, packagers will also remove Python-2-based plpython packages. So there is no issue. We could change plpythonu to be an alias for plpython3u instead of plpython2u, but that would be a PG13 (or later) change. I notice that PEP-394 now supports this. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Why would it be a 13 or later issue?
I am specifically talking about changing the default.
On Sat, Jul 6, 2019, 6:28 PM Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
On 2019-07-06 21:02, Steven Pousty wrote:
> /" The default will probably be changed to Python 3 in a distant future
> release of PostgreSQL, depending on the progress of the migration to
> Python 3 in the Python community."/
>
> I know we are late in the Postgresql 12 cycle but I think switching the
> default to Python 3 is warranted given:
"The default" in the above statement refers to what version the language
name "plpythonu" invokes. That is separate from whether an installer or
packager chooses to install a Python-2-based version. I expect that as
operating systems remove Python 2 from their systems, packagers will
also remove Python-2-based plpython packages. So there is no issue.
We could change plpythonu to be an alias for plpython3u instead of
plpython2u, but that would be a PG13 (or later) change. I notice that
PEP-394 now supports this.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2019-07-07 00:34, Steven Pousty wrote: > Why would it be a 13 or later issue? Because PostgreSQL 12 is feature frozen and in beta, and this issue is not a regression. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 2019-07-07 00:34, Steven Pousty wrote:
>> Why would it be a 13 or later issue?
> Because PostgreSQL 12 is feature frozen and in beta, and this issue is
> not a regression.
More to the point: it does not seem to me that we should change what
"plpythonu" means until Python 2 is effectively extinct in the wild.
Which is surely some years away yet. If we change it sooner than
that, the number of people complaining that we broke perfectly good
installations will vastly outweigh the number of people who are
happy because we saved them one keystroke per function definition.
As a possibly relevant comparison, I get the impression that most
packagers of Python are removing the versionless "python" executable
name and putting *nothing* in its place. You have to write python2
or python3 nowadays. Individuals might still be setting up symlinks
so that "python" does what they want, but it's not happening at the
packaging/distro level.
(This comparison suggests that maybe what we should be thinking
about is a way to make it easier to change what "plpythonu" means
at the local-opt-in level.)
regards, tom lane
The point of the links I sent from the Python community is that they wanted Python extinct in the wild as of Jan 1 next year. They are never fixing it, even for a security vulnerability.
It seems to me we roll out breaking changes with major versions. So yes, if the user chooses to upgrade to 12 and they haven't migrated their code to Python 2 it might not work.
I don't have a good answer to no changes except regressions. I do hope, given how much our users expect us to be secure, that we weigh the consequences of making our default Python a version which is dead to the community a month or so after Postgresql 12s release. We can certainly take the stance of leave the Python version be, but it seems that we should then come up with a plan if there is a security vulnerability found in Python 2 after Jan 1st 2020.
If Python 2 wasn't our default choice then I would be much more comfortable letting this just pass without mention.
All that aside, I think allowing the admin set the default version of plpythonu to be an excellent idea.
Thanks
Steve
On Sun, Jul 7, 2019, 8:26 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 2019-07-07 00:34, Steven Pousty wrote:
>> Why would it be a 13 or later issue?
> Because PostgreSQL 12 is feature frozen and in beta, and this issue is
> not a regression.
More to the point: it does not seem to me that we should change what
"plpythonu" means until Python 2 is effectively extinct in the wild.
Which is surely some years away yet. If we change it sooner than
that, the number of people complaining that we broke perfectly good
installations will vastly outweigh the number of people who are
happy because we saved them one keystroke per function definition.
As a possibly relevant comparison, I get the impression that most
packagers of Python are removing the versionless "python" executable
name and putting *nothing* in its place. You have to write python2
or python3 nowadays. Individuals might still be setting up symlinks
so that "python" does what they want, but it's not happening at the
packaging/distro level.
(This comparison suggests that maybe what we should be thinking
about is a way to make it easier to change what "plpythonu" means
at the local-opt-in level.)
regards, tom lane
On 2019-07-07 21:26, Steven Pousty wrote: > The point of the links I sent from the Python community is that they > wanted Python extinct in the wild as of Jan 1 next year. They are never > fixing it, even for a security vulnerability. The operating systems that most of our users are going to run PostgreSQL 12 on will keep maintaining Python 2 for the foreseeable future. I'm not trying to dismiss the importance of managing the Python transition. But this issue has been known for many years, and the current setup is more or less in line with the wider world. For example, the Debian release that came out over the weekend still ships with /usr/bin/python being Python 2. So it is neither timely nor urgent to try to make some significant change about this in PostgreSQL 12 right now. I would welcome patches for this for PostgreSQL 13. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> I'm not trying to dismiss the importance of managing the Python
> transition. But this issue has been known for many years, and the
> current setup is more or less in line with the wider world. For
> example, the Debian release that came out over the weekend still ships
> with /usr/bin/python being Python 2. So it is neither timely nor urgent
> to try to make some significant change about this in PostgreSQL 12 right
> now. I would welcome patches for this for PostgreSQL 13.
I don't think it's been mentioned in this thread yet, but we *did*
recently install a configure-time preference for python3 over python2:
Author: Peter Eisentraut <peter@eisentraut.org>
Branch: master Release: REL_12_BR [7291733ac] 2019-01-13 10:23:48 +0100
Branch: REL_11_STABLE Release: REL_11_2 [3d498c65a] 2019-01-13 10:24:21 +0100
Branch: REL_10_STABLE Release: REL_10_7 [cd1873160] 2019-01-13 10:25:23 +0100
configure: Update python search order
Some systems don't ship with "python" by default anymore, only
"python3" or "python2" or some combination, so include those in the
configure search.
Discussion:
https://www.postgresql.org/message-id/flat/1457.1543184081%40sss.pgh.pa.us#c9cc1199338fd6a257589c6dcea6cf8d
configure's search order is now $PYTHON, python, python3, python2.
I think it will be a very long time, if ever, before there would be
a reason to consider changing that. Both of the first two options
represent following a clear user preference.
So the only thing that's really at stake is when/whether we can make
"plpythonu" a synonym for "plpython3u" rather than "plpython2u".
As I said already, I think that's got to be a long way off, since the
whole problem here is that python3 isn't a drop-in replacement for
python2. We're much more likely to break existing functions than do
anything useful by forcibly switching the synonym.
But I could support having a way for individual installations to change
what the synonym means locally. Perhaps we could think about how to do
that in conjunction with the project of getting rid of pg_pltemplate
that's been kicked around before [1][2][3].
regards, tom lane
[1] https://www.postgresql.org/message-id/flat/763f2fe4-743f-d530-8831-20811edd3d6a%402ndquadrant.com
[2] https://www.postgresql.org/message-id/flat/7495.1524861244%40sss.pgh.pa.us
[3] https://www.postgresql.org/message-id/flat/5351890.TdMePpdHBD%40nb.usersys.redhat.com
I wrote:
> But I could support having a way for individual installations to change
> what the synonym means locally. Perhaps we could think about how to do
> that in conjunction with the project of getting rid of pg_pltemplate
> that's been kicked around before [1][2][3].
... actually, if we had that (i.e., languages fully defined by extensions
with no help from pg_pltemplate), wouldn't this be nearly trivial?
I'm imagining two extensions, one that defines plpythonu to call the
python2 code and one that defines it to call the python3 code, and you
install whichever you want. They're separate from the extensions that
define plpython2u and plpython3u, so mix and match as you wish.
regards, tom lane