Thread: What's our minimum supported Python version?
Our fine manual claims the answer to $SUBJECT is 3.2.
However, there is room to doubt that our code still actually
works with 3.2, because the oldest version I can find being
tested by the buildfarm is topminnow's 3.4.2. The next oldest
is shelduck's 3.4.10, and everything else has 3.5.2 or newer.
topminnow hasn't reported in a couple months, so it may
actually be dead.
The reason I bring this up is that I found out the hard way
that src/test/modules/oauth_validator fails on RHEL8, because
its oauth_server.py script is not compatible with the 3.6.8
version of Python supplied by this distro. (There are 22
buildfarm animals running 3.6.8, presumably mostly also Red
Hat-derived platforms, so I'm not going to apologize for
my workstation being a little long in the tooth.)
I think we need to do some combination of moving our
minimum-supported-version goalposts forward, making sure that
whatever we claim is the minimum Python version is actually
being tested in the buildfarm, and fixing oauth_server.py
so that it works on that version.
I am not familiar enough with the Python landscape to
have an informed opinion about what the minimum supported
version should be. But I'll present this data scraped
from the buildfarm about how many animals are running what
(counting animals that have reported since 2025-01-01):
Count Version
1 3.4.2
1 3.4.10
2 3.5.2
3 3.5.3
2 3.6.5
22 3.6.8
3 3.6.9
5 3.6.15
1 3.7.1
2 3.7.3
1 3.7.10
2 3.7.16
1 3.8.8
4 3.8.10
15 3.9.2
6 3.9.16
6 3.9.18
1 3.9.19
2 3.9.20
13 3.9.21
6 3.10.x
18 3.11.x
21 3.12.x
22 3.13.x
160 total
Thoughts?
regards, tom lane
> On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > Our fine manual claims the answer to $SUBJECT is 3.2. > > However, there is room to doubt that our code still actually > works with 3.2, because the oldest version I can find being > tested by the buildfarm is topminnow's 3.4.2. The next oldest > is shelduck's 3.4.10, and everything else has 3.5.2 or newer. > topminnow hasn't reported in a couple months, so it may > actually be dead. > > The reason I bring this up is that I found out the hard way > that src/test/modules/oauth_validator fails on RHEL8, because > its oauth_server.py script is not compatible with the 3.6.8 > version of Python supplied by this distro. (There are 22 > buildfarm animals running 3.6.8, presumably mostly also Red > Hat-derived platforms, so I'm not going to apologize for > my workstation being a little long in the tooth.) > > I think we need to do some combination of moving our > minimum-supported-version goalposts forward, making sure that > whatever we claim is the minimum Python version is actually > being tested in the buildfarm, and fixing oauth_server.py > so that it works on that version. > > I am not familiar enough with the Python landscape to > have an informed opinion about what the minimum supported > version should be. But I'll present this data scraped > from the buildfarm about how many animals are running what > (counting animals that have reported since 2025-01-01): > > Count Version > > 1 3.4.2 > 1 3.4.10 > 2 3.5.2 > 3 3.5.3 > 2 3.6.5 > 22 3.6.8 > 3 3.6.9 > 5 3.6.15 > 1 3.7.1 > 2 3.7.3 > 1 3.7.10 > 2 3.7.16 > 1 3.8.8 > 4 3.8.10 > 15 3.9.2 > 6 3.9.16 > 6 3.9.18 > 1 3.9.19 > 2 3.9.20 > 13 3.9.21 > 6 3.10.x > 18 3.11.x > 21 3.12.x > 22 3.13.x > > 160 total > > Thoughts? > > regards, tom lane > From an Python ecosystem perspective, 3.9 is the usual minimum that people use in CI matrices nowdays. So if it was up to me, that’s what I’d choose. Mabe 3.7 if I was too liberal, but <3.6 and older it’s probably too old to bother. Then again, I’m not a buildfarm power user, so my 2 cents are worth a bit less.
Florents Tselai <florents.tselai@gmail.com> writes:
> On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I think we need to do some combination of moving our
>> minimum-supported-version goalposts forward, making sure that
>> whatever we claim is the minimum Python version is actually
>> being tested in the buildfarm, and fixing oauth_server.py
>> so that it works on that version.
> From an Python ecosystem perspective,
> 3.9 is the usual minimum that people use in CI matrices nowdays.
> So if it was up to me, that’s what I’d choose.
Per these numbers, that would be cutting off 31% of the buildfarm,
including a lot of still-in-support distros such as RHEL8.
So I would say that's not likely to be our choice.
regards, tom lane
> On 19 Apr 2025, at 18:17, Tom Lane <tgl@sss.pgh.pa.us> wrote: > The reason I bring this up is that I found out the hard way > that src/test/modules/oauth_validator fails on RHEL8, because > its oauth_server.py script is not compatible with the 3.6.8 > version of Python supplied by this distro. Do you have the error message/log for the failure handy? -- Daniel Gustafsson
Daniel Gustafsson <daniel@yesql.se> writes:
>> On 19 Apr 2025, at 18:17, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> The reason I bring this up is that I found out the hard way
>> that src/test/modules/oauth_validator fails on RHEL8, because
>> its oauth_server.py script is not compatible with the 3.6.8
>> version of Python supplied by this distro.
> Do you have the error message/log for the failure handy?
The first problem is that this Python version seems not to
like assignments embedded in if statements:
File "t/oauth_server.py", line 319
if err := self._get_param("error_code", None):
^
SyntaxError: invalid syntax
I was able to work around that with the attached quick hack.
But then I get
Traceback (most recent call last):
File "t/oauth_server.py", line 19, in <module>
class OAuthHandler(http.server.BaseHTTPRequestHandler):
File "t/oauth_server.py", line 26, in OAuthHandler
JsonObject = dict[str, object] # TypeAlias is not available until 3.10
TypeError: 'type' object is not subscriptable
which I have no idea how to work around.
regards, tom lane
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
index 5bc30be87fd..133fe496cfc 100755
--- a/src/test/modules/oauth_validator/t/oauth_server.py
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -316,11 +316,13 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
return resp
def token(self) -> JsonObject:
- if err := self._get_param("error_code", None):
+ err = self._get_param("error_code", None)
+ if err:
self._response_code = self._get_param("error_status", 400)
resp = {"error": err}
- if desc := self._get_param("error_desc", ""):
+ desc = self._get_param("error_desc", "")
+ if desc:
resp["error_description"] = desc
return resp
On Tue, Apr 22, 2025 at 8:05 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > The first problem is that this Python version seems not to > like assignments embedded in if statements: [...] > > I was able to work around that with the attached quick hack. > But then I get [...] Thanks, I'll put a patch together. Sorry -- IIRC, both of those features are probably too new for me to have used, regardless of what we decide is the minimum version for 18. As for picking a version... 3.6 will have been EOL for almost three years by the time 18 releases. It seems like we would drop it happily, were it not for RHEL8. But RHEL also supports side-by-side Python, right? In general, I'd be more than willing to try plumbing that through the new tests, if it lets us drop end-of-life versions for new code. (I don't know if we'll need to drop 3.6 for PG18, specifically, but I also think we shouldn't plan to support 3.6 for the full RHEL8 lifetime.) Thanks, --Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> As for picking a version... 3.6 will have been EOL for almost three
> years by the time 18 releases. It seems like we would drop it happily,
> were it not for RHEL8.
Agreed, but RHEL8 is out there and I don't think we can just drop
support for it. I'm also not excited by the idea that an incidental
test script gets to dictate what the cutoff is.
I do think we should stop claiming that Python 3.2 will work.
(Maybe it does, but we don't know that.) I see that the configure
script only checks for Python >= 3, and it doesn't look like the
meson scripts check explicitly at all, although there's a comment
saying that our meson version cutoff is intended to allow working
with Python 3.5.
Maybe it's sufficient to make a documentation change here, and
say we support Python >= 3.5? I'd be okay with saying 3.6.8
too, on the grounds that if anything older fails to work we'd
almost certainly just say "too bad". But RHEL8 is widespread
enough that I think we need to keep making the effort for 3.6.8.
regards, tom lane
On Tue, Apr 22, 2025 at 9:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > I'm also not excited by the idea that an incidental > test script gets to dictate what the cutoff is. I agree, and I don't intend for the script to dictate that. > Maybe it's sufficient to make a documentation change here, and > say we support Python >= 3.5? I'd be okay with saying 3.6.8 > too, on the grounds that if anything older fails to work we'd > almost certainly just say "too bad". I'm definitely on board with pulling the minimum up, as high as we think we can get away with. If you're comfortable with 3.6(.8), I'd say let's start there. > But RHEL8 is widespread > enough that I think we need to keep making the effort for 3.6.8. Even if we can get side-by-side versions working? RHEL8 has a bunch of newer Python 3 versions available, according to the docs. And virtualenvs can do a lot of lifting for us in practice. Thanks, --Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Tue, Apr 22, 2025 at 9:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> But RHEL8 is widespread
>> enough that I think we need to keep making the effort for 3.6.8.
> Even if we can get side-by-side versions working?
That would be off-loading our problem onto the users (and the
buildfarm owners). If we really had little other choice,
we could go there; but it doesn't sound like this is that
hard to work around.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes: > Florents Tselai <florents.tselai@gmail.com> writes: >> On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> I think we need to do some combination of moving our >>> minimum-supported-version goalposts forward, making sure that >>> whatever we claim is the minimum Python version is actually >>> being tested in the buildfarm, and fixing oauth_server.py >>> so that it works on that version. > >> From an Python ecosystem perspective, >> 3.9 is the usual minimum that people use in CI matrices nowdays. >> So if it was up to me, that’s what I’d choose. > > Per these numbers, that would be cutting off 31% of the buildfarm, > including a lot of still-in-support distros such as RHEL8. > So I would say that's not likely to be our choice. > > regards, tom lane Also from a python world perspective, we really don't want to run EOL versions. Besides the security reasons, bugs in thirdy-party dependencies usually are not fixed and the whole system seems to rot very quickly. Of course, this does not happen if one does not rely on third-party deps. Even in this case, I would say that it is wise to support a relatively small set because the built-in libs will vary over versions, and a wide range of versions will cause more troubles like this one. The oldest non EOL version is 3.9 right now. My suggestion is to follow the official supported releases. I'm not familiar with the buildfarm, but I know that python3.6 does not come installed by default in RHEL8, so instead of installing this version we can simply install 3.9, 3.11 or 3.12. In a persistent workstation, I believe the following should do the trick (tested on a 8.5 box): sudo dnf install python3.12 sudo dnf remove python36 BTW, we are used to create a virtualenv for everything, but that is another topic. Best regards, Renan Fonseca
On Tue, Apr 22, 2025 at 8:29 AM Jacob Champion <jacob.champion@enterprisedb.com> wrote: > On Tue, Apr 22, 2025 at 8:05 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > > The first problem is that this Python version seems not to > > like assignments embedded in if statements: [...] > > > > I was able to work around that with the attached quick hack. > > But then I get [...] > > Thanks, I'll put a patch together. Sorry -- IIRC, both of those > features are probably too new for me to have used, regardless of what > we decide is the minimum version for 18. Splitting this off into its own $SUBJECT. I coupled oauth_server to new Python stuff without realizing it, sorry: - urllib's reclassification of `~` as a safe URL character (3.7) - walrus operator `:=` (3.8) - dict[type, type] annotations (3.9) - str.removeprefix/suffix() (3.9) Attached patch gets rid of all that. This is tested against Python 3.6.15 (3.6 went EOL at the end of 2021). I'm working on getting Rocky 8 installed locally to test against. If it's decided we want to support downwards to 3.5, I will test there too (but I hope we don't; see parent thread). Thanks, --Jacob
Attachment
On Tue, Apr 22, 2025 at 12:09 PM Renan Alves Fonseca <renanfonseca@gmail.com> wrote: > The oldest non EOL version is 3.9 right now. My suggestion is to follow > the official supported releases. I think this policy is too aggressive. Many operating systems we support are going to ship Python versions past their EOL date (and keep them supported for a long while with security patches). There's a middle ground to be found, IMO. --Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> This is tested against Python 3.6.15 (3.6 went EOL at the end of
> 2021). I'm working on getting Rocky 8 installed locally to test
> against. If it's decided we want to support downwards to 3.5, I will
> test there too (but I hope we don't; see parent thread).
Thanks! I confirm this works for me on RHEL8 with 3.6.8.
regards, tom lane
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Tue, Apr 22, 2025 at 12:09 PM Renan Alves Fonseca
> <renanfonseca@gmail.com> wrote:
>> The oldest non EOL version is 3.9 right now. My suggestion is to follow
>> the official supported releases.
> I think this policy is too aggressive. Many operating systems we
> support are going to ship Python versions past their EOL date (and
> keep them supported for a long while with security patches).
Yeah, that. The distros that are still shipping older Pythons
are LTS distros where part of the deal is that the vendor
is supposed to back-patch security fixes, even if the upstream
has moved on. Maybe a particular vendor is falling down on that
job, but it's not for us to rate their performance. So I don't
buy that "but security!" is a good argument here.
(Full disclosure: I worked for Red Hat for more than a decade,
and still preferentially use their distros. So I've pretty well
bought into that model.)
regards, tom lane
On Tue, Apr 22, 2025 at 12:36 PM Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thanks! I confirm this works for me on RHEL8 with 3.6.8. Great, thanks for the quick review! I'm considering committing this myself. Do you mind if it takes a day or so to get this in, while I quadruple-check everything, or are you blocked on this? --Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> I'm considering committing this myself.
Please do.
> Do you mind if it takes a day
> or so to get this in, while I quadruple-check everything, or are you
> blocked on this?
I'm not in a hurry. Push when you're ready.
regards, tom lane
tl;dr I think requiring support of Python 3.9 for PG18 would probably be reasonable and save us a bunch of maintenance burden over the next 5 years. On Tue, 22 Apr 2025 at 21:41, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Yeah, that. The distros that are still shipping older Pythons > are LTS distros where part of the deal is that the vendor > is supposed to back-patch security fixes, even if the upstream > has moved on. Maybe a particular vendor is falling down on that > job, but it's not for us to rate their performance. So I don't > buy that "but security!" is a good argument here. I totally agree that "security" is not the discussion that we should be having here. I think the Python version decision (or really any minimum version decision) should be based on some analysis of costs to us for maintaining support vs benefits to the users. I'm pretty sure most users of RHEL expect most modern software not to compile/work completely effortlessly on their distros without some effort on their part or on the part of RHEL packagers. That's kinda what you're signing up for if you choose a distro like that. It might very well be that supporting Python 3.6 does not require many code changes. But it's so old that installing it on most of the recent OSes is not trivial, e.g. uv does not support installing it[1]. By saying we support it, it means that all developers need to go through serious hoops to be able to test that their python scripts work correctly on their own machines. And given our own support policy, we'd probably have to do that for 5 more years (unless we want to change supported python versions in minor PG releases, which I highly doubt we want). So that leaves the question, how much are we actually helping users by spending all that effort on keeping support for old Python versions? Based on the Red Hat docs, it's pretty easy to install newer Python3 versions on RHEL8. Depending on the X in RHEL 8.X you can install different Python3 versions from the official repos[2]. And based on the official RHEL EOL policy[3], only RHEL 8.4, 8.6, 8.8 and 8.10 are still getting some level of support. With 8.4 and 8.6 only getting the final level of support called "Update Services for SAP Solutions", and with 8.8 reaching that same final support level before PG18 is released (and 8.4 becoming unsupported before PG18). Based on that information (only taking into account only RHEL8, not other distros). It seems like we could at least bump the minimum Python support to Python 3.9, which is the newest python that's installable in RHEL 8.4 and 8.6. Anyone that's running a RHEL version that's still supported by Red Hat can then simply do "yum install python39" and run the tests (assuming we set up meson/autoconf to correctly detect the "python3.9" binary). I'd even be curious how much resistance we'd get from RHEL users if we'd bump the Python requirement for PG18 to Python 3.12, which is the newest version that RHEL 8.10 supports. RHEL 8.10 is the only RHEL 8 version that is getting regular support from Red Hat. (Full disclosure: I've used close to EOL versions of RHEL7 professionally, and I hated every second it. I constantly felt like I was walking through a software museum. Having to work around bugs that had been fixed for ages in upstream) Looking a little further than RHEL: - Ubuntu 20.04 can easily install Python3.9 from the official package repos. Currently this is still supported by Cristoph his pgdg repos, but looks like that support might be removed before PG18 given it's transition to "Expanded Security Maintenance" next month. - Ubuntu 22.04 can easily install Python3.11 from the official package repos. Still has full upstream support for 2 years. - Debian Bullseye (EOL in 1.5 years) can only install Python3.9 from the official package repos. - Debian Bookworm (EOL in 3 years) has Pytho - Python 3.8 itself is declared EOL by upstream - Python 3.10 will be declared EOL by upstream around the release of PG19 Based on all this info, I think that it sounds quite reasonable to start requiring Python3.9 for PG18. And for PG19 we could probably start requiring Python3.11 [1]: https://github.com/astral-sh/uv/issues/9833 [2]: https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/installing-and-using-dynamic-programming-languages_configuring-basic-system-settings#assembly_installing-and-using-python_installing-and-using-dynamic-programming-languages [3]: https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide P.S. I CC-ed Devrim and Christoph, for insights into this from the packagers perspective.
Re: Jelte Fennema-Nio
> - Ubuntu 20.04 can easily install Python3.9 from the official package
> repos. Currently this is still supported by Cristoph his pgdg repos,
> but looks like that support might be removed before PG18 given it's
> transition to "Expanded Security Maintenance" next month.
20.04 (focal) is already too old for PG18 because of llvm version
woes, so we are already not building it there. (I forgot what the
precise problem was, but I figured focal would be EOL before the PG18
release anyway.)
> - Ubuntu 22.04 can easily install Python3.11 from the official package
> repos. Still has full upstream support for 2 years.
> - Debian Bullseye (EOL in 1.5 years) can only install Python3.9 from
> the official package repos.
> - Debian Bookworm (EOL in 3 years) has Pytho
^ n 3.11
> - Python 3.8 itself is declared EOL by upstream
> - Python 3.10 will be declared EOL by upstream around the release of PG19
> Based on all this info, I think that it sounds quite reasonable to
> start requiring Python3.9 for PG18. And for PG19 we could probably
> start requiring Python3.11
Ack. Bullseye LTS will be supported until August 2026, so very close
to the PG19 release, but it should work out not to support PG19 there.
https://wiki.debian.org/LTS/
Christoph
On Tue, Apr 22, 2025 at 2:28 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote: > I'm pretty sure most users of RHEL expect most modern software not to > compile/work completely effortlessly on their distros without some > effort on their part or on the part of RHEL packagers. That's kinda > what you're signing up for if you choose a distro like that. This is the core of my position, too. I think it's reasonable to support Python versions for some time after they go EOL, but I don't think that we need to treat "users who want bleeding-edge Postgres with long-EOL dependencies" as something to cater to, at the expense of the committer testing matrix. Note that Meson itself has updated to Python 3.7 as a minimum version (as it now warns you, loudly). Thanks, --Jacob
Jelte Fennema-Nio <postgres@jeltef.nl> writes:
> I totally agree that "security" is not the discussion that we should
> be having here. I think the Python version decision (or really any
> minimum version decision) should be based on some analysis of costs to
> us for maintaining support vs benefits to the users.
[ shrug... ] This thread in itself has already consumed more effort
than we've spent thinking about old-Python problems for the last
several years. (Otherwise, we would probably have updated that
supported-version claim in installation.sgml some time ago.)
So I doubt that there's all that much in cost savings to us that'd
outweigh benefits to users. The fact that Jacob was able to fix
oauth_server.py without (it seemed like) very much work seems to
bear out that opinion.
This calculus might change if we start to rely more heavily on
Python for build and test infrastructure than we do today. But
I think we can have that conversation when it happens.
Worth noting also is that there are different requirements in
different places. For example,
contrib/unaccent/generate_unaccent_rules.py is probably only ever run
by developers, so I doubt anyone is going to make a fuss if it doesn't
work on hoary Pythons. Test infrastructure is more of a problem,
since we want to be able to test on any platform that we consider
supported at all.
Anyway, what I propose that we do for now is replace the
installation.sgml text
The minimum required version is Python 3.2.
with
The minimum supported version is Python 3.6.8.
where I use "supported" advisedly, as in "it might work with
something older, but we don't promise to fix it if not".
regards, tom lane
On Tue, 22 Apr 2025 at 21:26, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> - str.removeprefix/suffix() (3.9)
The way you replaced this does not have the same behaviour in the case
where the prefix/suffix is not part of the string. removeprefix/suffix
will not remove any characters in that case, but your code will always
remove the number of characters that the suffix/prefix is long. Maybe
your other checks and definition of the OAuth spec ensure that these
prefixes/suffixes are present when you remove them, but the code is
not the same in the general case (you'd need to add a hasprefix/suffix
check before removing the characters.
+ # Strip off the magic path segment. (The more readable
+ # str.removeprefix()/removesuffix() aren't available until Py3.9.)
if self._alt_issuer:
# The /alternate issuer uses IETF-style .well-known URIs.
if self.path.startswith("/.well-known/"):
- self.path = self.path.removesuffix("/alternate")
+ self.path = self.path[: -len("/alternate")]
else:
- self.path = self.path.removeprefix("/alternate")
+ self.path = self.path[len("/alternate") :]
elif self._parameterized:
- self.path = self.path.removeprefix("/param")
+ self.path = self.path[len("/param") :]
On Tue, Apr 22, 2025 at 3:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote: > The fact that Jacob was able to fix > oauth_server.py without (it seemed like) very much work seems to > bear out that opinion. I think my ability to do it should not be used as evidence of "ease". I knew where to download 3.6, that I should build it with Clang to avoid GCC segfaults, that I should set the rpaths appropriately, and to wrap it all in a virtualenv for Meson to find. And I had to kick Meson to get it to clear its cached Python information, or else the contrib modules refused to build... I suspect the only way most developers are going to want to test this is by running EL8. > This calculus might change if we start to rely more heavily on > Python for build and test infrastructure than we do today. But > I think we can have that conversation when it happens. As long as the need to backport to PG18 doesn't freeze that conversation in place, I suppose. > Anyway, what I propose that we do for now is replace the > installation.sgml text > > The minimum required version is Python 3.2. > > with > > The minimum supported version is Python 3.6.8. > > where I use "supported" advisedly, as in "it might work with > something older, but we don't promise to fix it if not". No objection to that as a first step. I'm still hopeful that Devrim has some evidence in favor of bumping to 3.8 or 3.9. :) Thanks, --Jacob
On Tue, Apr 22, 2025 at 3:17 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote: > The way you replaced this does not have the same behaviour in the case > where the prefix/suffix is not part of the string. removeprefix/suffix > will not remove any characters in that case, but your code will always > remove the number of characters that the suffix/prefix is long. Maybe > your other checks and definition of the OAuth spec ensure that these > prefixes/suffixes are present when you remove them Correct. Directly above the changed code are the prefix checks, which set self._alt_issuer/_parameterized. Thanks! --Jacob
Hi, On Tue, 2025-04-22 at 15:33 -0700, Jacob Champion wrote: > I'm still hopeful that Devrim has some evidence in favor of bumping > to 3.8 or 3.9. :) I would love to have such an evidence -- but I don't have :) In the last couple of weeks I've also been thinking of bumping every single Python piece in the PGDG RPM repository to 3.9 (at least) on RHEL 8, but that will break many things immediately. It is still a very major platform for users and such a breakage is not welcome. Regards, -- Devrim Gündüz Open Source Solution Architect, PostgreSQL Major Contributor BlueSky: @devrim.gunduz.org , @gunduz.org
Attachment
On Wed, 23 Apr 2025 at 10:15, Devrim Gündüz <devrim@gunduz.org> wrote: > I would love to have such an evidence -- but I don't have :) In the last > couple of weeks I've also been thinking of bumping every single Python > piece in the PGDG RPM repository to 3.9 (at least) on RHEL 8, but that > will break many things immediately. It is still a very major platform > for users and such a breakage is not welcome. Sad... Sounds like a bump to 3.9 is probably out of the question for PG18 then. Can you share some examples of the things that break? Because if (and I'm hopeful that Jacob will be adding some) for PG19 we get more testing infrastructure in Python, I wouldn't want all that extra testing infra to be stuck on Python3.6 for the next 5 years. I think the UX that we should make work for compiling/testing infra on RHEL8 is the following: yum install python39 ./configure PYTHON=python39 # or equivalent meson make check-world That seems like a reasonable hurdle for people using RHEL8 to jump over if they want to compile themselves. Especially since when installing from pgdg-rpm the package could simply depend on the python39 package. I haven't tried, but I'm pretty sure we're not there yet. At minimum I believe we'd have to change our shebang's from: #!/usr/bin/env python3 To something that takes into account the passed in PYTHON option.
Hi, On Wed, 2025-04-23 at 10:47 +0200, Jelte Fennema-Nio wrote: > Can you share some examples of the things that break? The most notable one would be Psycopg (2 and 3). Plus Patroni dependencies. There may be a couple of more. Regards, -- Devrim Gündüz Open Source Solution Architect, PostgreSQL Major Contributor BlueSky: @devrim.gunduz.org , @gunduz.org
Attachment
On Wed, 23 Apr 2025 at 10:59, Devrim Gündüz <devrim@gunduz.org> wrote: > The most notable one would be Psycopg (2 and 3). Plus Patroni > dependencies. There may be a couple of more. I meant more in what ways do these things break? Since you're actually the one that's packaging them, I'd expect that you could make them depend on the python39 package and be mostly done.
On Wed, 23 Apr 2025 at 00:33, Jacob Champion <jacob.champion@enterprisedb.com> wrote: > As long as the need to backport to PG18 doesn't freeze that > conversation in place, I suppose. I'm confused. Are you intending to backport new test infra to PG18? Looking at the amount of python code that we currently have, I agree with Tom: Making the few scripts that we have compatible with Python3.6 seems the best solution for PG18. (especially since you already have a patch that fixes the known issues). Given the purpose and pretty small size of the scripts, I expect that it'd be extremely rare for us to backport changes to them. e.g. I doubt your oauth server would need many changes to keep working correctly for the lifetime of PG18. Maybe it gets some minor fixes in 18.1 and/or 18.2 but other than that it seems really unlikely.
Hi, On Wed, 2025-04-23 at 11:03 +0200, Jelte Fennema-Nio wrote: > > I meant more in what ways do these things break? Since you're actually > the one that's packaging them, I'd expect that you could make them > depend on the python39 package and be mostly done. You are right, our side is fixable. However many packages in the upstream also depend on Psycopg. I don't want to create a Linux distribution based on RHEL 8 built against Python 3.9 (or 3.1x) :-) Regards, -- Devrim Gündüz Open Source Solution Architect, PostgreSQL Major Contributor BlueSky: @devrim.gunduz.org , @gunduz.org
Attachment
On Wed, 23 Apr 2025 at 11:13, Devrim Gündüz <devrim@gunduz.org> wrote: > You are right, our side is fixable. However many packages in the > upstream also depend on Psycopg. I don't want to create a Linux > distribution based on RHEL 8 built against Python 3.9 (or 3.1x) :-) I'm confused. The upstream RHEL8 repo depends on packages in pgdg??? Or are you saying that psycopg (and patroni) are not part of the pgdg-repo, and instead part of the upstream-repo. And so that other packages in pgdg that depend on upstream psycopg, would always need to support python 3.6 (because that's what upstream psycopg uses).
Hi, On Wed, 2025-04-23 at 11:46 +0200, Jelte Fennema-Nio wrote: > I'm confused. The upstream RHEL8 repo depends on packages in pgdg??? > > Or are you saying that psycopg (and patroni) are not part of the > pgdg-repo, and instead part of the upstream-repo. And so that other > packages in pgdg that depend on upstream psycopg, would always need to > support python 3.6 (because that's what upstream psycopg uses). psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs 2.9.5) so they override OS packages. That is why things will break. A solution would be creating our own psycopg2 (likely for Python 3.12) package, update all PGDG packages that depend on psycopg2 to use that package. That is not impossible (I have already psycopg2 built against Python 3.11 on SLES 15), but I don't know how much work it will be and its impact as that set of update should go to both RHEL 8, 9 and 10 (RHEL 10 already includes Python 3.12 by default) I can go for this solution if it is *absolutely* required. We already have custom packages to support PostGIS, so that is not a new topic for me. Regards, -- Devrim Gündüz Open Source Solution Architect, PostgreSQL Major Contributor BlueSky: @devrim.gunduz.org , @gunduz.org
Attachment
On Wed, 23 Apr 2025 at 13:24, Devrim Gündüz <devrim@gunduz.org> wrote: > psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs > 2.9.5) so they override OS packages. That is why things will break. > > A solution would be creating our own psycopg2 (likely for Python 3.12) > package, update all PGDG packages that depend on psycopg2 to use that > package. That is not impossible (I have already psycopg2 built against > Python 3.11 on SLES 15), but I don't know how much work it will be and > its impact as that set of update should go to both RHEL 8, 9 and 10 > (RHEL 10 already includes Python 3.12 by default) Thanks for that explanation, now I understand the problem. > I can go for this solution if it is *absolutely* required. We already > have custom packages to support PostGIS, so that is not a new topic for > me. I don't really think this would be required to bump Postgres its minimum supported postgres version. psycopg is a client and postgresql is the server, it seems fine for them to use different Python versions. The postgresql-server package could depend on python3.9, while psycopg packages would happily use python3.6.
On Tue, Apr 22, 2025 at 12:45 PM Jacob Champion <jacob.champion@enterprisedb.com> wrote: > Great, thanks for the quick review! And pushed. Thanks everyone! --Jacob
> On 23 Apr 2025, at 22:26, Jacob Champion <jacob.champion@enterprisedb.com> wrote: > > On Tue, Apr 22, 2025 at 12:45 PM Jacob Champion > <jacob.champion@enterprisedb.com> wrote: >> Great, thanks for the quick review! > > And pushed. Thanks everyone! Congratulations on your first commit, looking forward to many more to come! -- Daniel Gustafsson
On Wed, Apr 23, 2025 at 4:24 AM Devrim Gündüz <devrim@gunduz.org> wrote: > psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs > 2.9.5) so they override OS packages. That is why things will break. Thanks, this is super helpful! --Jacob
On Wed, Apr 23, 2025 at 2:11 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote: > I'm confused. Are you intending to backport new test infra to PG18? Not particularly (though, if the barriers were low enough, wouldn't that be nice?). I'm talking about hazards in oauth_server.py here. > Given the purpose and pretty small size of the scripts, I expect that > it'd be extremely rare for us to backport changes to them. e.g. I > doubt your oauth server would need many changes to keep working > correctly for the lifetime of PG18. Maybe it gets some minor fixes in > 18.1 and/or 18.2 but other than that it seems really unlikely. That assumes there are no bugs in my code to backport tests for, and I don't like assuming that. The Perl and Python sides of the oauth_validator tests are pretty tightly coupled out of necessity. On Wed, Apr 23, 2025 at 7:54 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote: > I don't really think this would be required to bump Postgres its > minimum supported postgres version. psycopg is a client and postgresql > is the server, it seems fine for them to use different Python > versions. The postgresql-server package could depend on python3.9, > while psycopg packages would happily use python3.6. I'm not sure that's the mental model we would want, since clients can be used server-side. Maybe a safer way to split, if and when we want to decouple this on RHEL, would be between clients of only the Python language (tests), and clients of the Python ABI (PL/Python, psycopg, etc.). Like, imagine a world where backported tests could use new goodies, but extensions remained solidly on platform defaults. That seems kind of nice to me, at a surface level. The Python stable ABI may change the calculus, too, but I'm not going to look into that right now... --Jacob
On 22.04.25 18:04, Tom Lane wrote: > Jacob Champion <jacob.champion@enterprisedb.com> writes: >> As for picking a version... 3.6 will have been EOL for almost three >> years by the time 18 releases. It seems like we would drop it happily, >> were it not for RHEL8. > > Agreed, but RHEL8 is out there and I don't think we can just drop > support for it. I'm also not excited by the idea that an incidental > test script gets to dictate what the cutoff is. > > I do think we should stop claiming that Python 3.2 will work. > (Maybe it does, but we don't know that.) I see that the configure > script only checks for Python >= 3, and it doesn't look like the > meson scripts check explicitly at all, although there's a comment > saying that our meson version cutoff is intended to allow working > with Python 3.5. > > Maybe it's sufficient to make a documentation change here, and > say we support Python >= 3.5? I'd be okay with saying 3.6.8 > too, on the grounds that if anything older fails to work we'd > almost certainly just say "too bad". But RHEL8 is widespread > enough that I think we need to keep making the effort for 3.6.8. There are a lot of different things that are meant by Python support nowadays. It used to be the case that the minimum Python version was (1) the oldest version that was required to get plpython to compile, and (2) the range of versions for which we would maintain "expected" files for the plpython tests (see history of src/pl/plpython/expected/README). #2 isn't really a problem anymore, it seems. It used to require yearly attention, but not anymore. (Maybe Python itself has gotten more stable, or we have changed our tests to be less sensitive to this, probably a combination of both.) #1 is also less of a hot-spot, as indicated that we still claim to support back to Python 3.2. Also, starting in PG18, we are using the Python "Limited API", so this is being enforced on the Python side. That means, in principle, if plpython compiles successfully right now on Python 3.13, then it should also compile successfully on Python 3.2. But now the new meanings are also, (3) what version of Python is required by the oldest supported Meson version, and (4) what version of Python can be assumed by various build and test scripts. There is actually (4a) scripts that are only run from a meson build, which should surely align with #3, and (4b) scripts that are also run by a make build, which is what oauth_server.py is, for which we can pick anything. The answer to #3 is currently Python 3.5 (see top of top-level meson.build). And then there is meaning (5) what version has anyone actually tested. Note that #2, #3, and #4 are build-time and #1 also has a run-time component. It would be plausible to say that you need a certain newer Python to build, but plpython can still run on older versions. If we were to make any changes, we need to make sure that the documentation is precise about this and matches the code (see #define Py_LIMITED_API). The cut-off in practice for these things is usually RHEL. PG18 currently still supports RHEL 7, which appears to come with Python 3.6. Seeing that the small problem with the test script was easily fixed, I think we should stick with that for now. There might be value in dropping support for RHEL 7 sometime, but that should be done across the board (meson version, openssl version, perl version, etc.), and requires some buildfarm changes, so I wouldn't do this right now.
On Thu, 24 Apr 2025 at 10:54, Peter Eisentraut <peter@eisentraut.org> wrote: > The cut-off in practice for these things is usually RHEL. PG18 > currently still supports RHEL 7, which appears to come with Python 3.6. > Seeing that the small problem with the test script was easily fixed, I > think we should stick with that for now. There might be value in > dropping support for RHEL 7 sometime, but that should be done across the > board (meson version, openssl version, perl version, etc.), and requires > some buildfarm changes, so I wouldn't do this right now. RHEL7 is not supported by Devrim for PG16 and PG17 already[1]. Did you mean something different with "PG18 currently still supports RHEL 7"? [1]: https://yum.postgresql.org/news/rhel7-postgresql-rpms-end-of-life/
On 24.04.25 13:16, Jelte Fennema-Nio wrote: > On Thu, 24 Apr 2025 at 10:54, Peter Eisentraut <peter@eisentraut.org> wrote: >> The cut-off in practice for these things is usually RHEL. PG18 >> currently still supports RHEL 7, which appears to come with Python 3.6. >> Seeing that the small problem with the test script was easily fixed, I >> think we should stick with that for now. There might be value in >> dropping support for RHEL 7 sometime, but that should be done across the >> board (meson version, openssl version, perl version, etc.), and requires >> some buildfarm changes, so I wouldn't do this right now. > > RHEL7 is not supported by Devrim for PG16 and PG17 already[1]. Did you > mean something different with "PG18 currently still supports RHEL 7"? > > [1]: https://yum.postgresql.org/news/rhel7-postgresql-rpms-end-of-life/ There are approximately 6 buildfarm members with RHEL 7 or CentOS 7, so in that sense, "support" means that everything is expected to work there. And some amount of work has been done recently to keep that support alive. (But I'm confused now because I see Python 3.6 on both the RHEL 7 and the RHEL 8 animals. So it's not clear how representative these animals are especially with respect to the Python versions.)
Peter Eisentraut <peter@eisentraut.org> writes:
> There are approximately 6 buildfarm members with RHEL 7 or CentOS 7, so
> in that sense, "support" means that everything is expected to work
> there. And some amount of work has been done recently to keep that
> support alive.
Yeah. Devrim's choice of what to package is not project policy,
it's just his own personal allocation of resources.
> (But I'm confused now because I see Python 3.6 on both the RHEL 7 and
> the RHEL 8 animals. So it's not clear how representative these animals
> are especially with respect to the Python versions.)
Per wikipedia, RHEL7 was released mid-2014, so the latest Python
version 7.0 could possibly have shipped with is 3.4 (released
2014-03) and much more likely they shipped 3.3 (2012-09). However,
you can if you choose install later Python versions, and you
have control over which version /usr/bin/python3 points at.
(At least this is true on the RHEL8 box I am looking at, and
I'm fairly sure it was so on RHEL7 too.) Similarly, Python 3.6
seems to be the baseline default on RHEL8 --- and the timing
more or less matches up, as 3.7 was released not long before
they would have been freezing the RHEL8 feature set. But you can
install 3.8, 3.9, 3.11, and/or 3.12 without going outside the
universe of Red-Hat-supported packages.
So what's that mean for us? "We still want to support RHEL7" turns
out to be squishier than one might think. But I don't think we
really want to promise to work with Python 3.3, especially given
the lack of any buildfarm representation of that. In the other
direction, yeah we could insist that RHEL users install some
later Python version, but I think we'd get push-back. The point
of using an LTS distro is, well, stability. The users want to
decide which packages they are comfortable with updating.
I'm still content with the idea of deciding that 3.6 is now our
cutoff. If someone comes along and complains because
oauth_server.py doesn't work on an older version, it's up to them
to provide a patch. If/when we want to include some Python code
that can't reasonably be made to work on 3.6, we can reconsider.
regards, tom lane
On Thu, Apr 24, 2025 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > I'm still content with the idea of deciding that 3.6 is now our > cutoff. Seems like no one is pushing hard for an earlier version, yet, so here's a patch with your suggested wording from upthread. I'm not sure if this meets Peter's request for precision. (Though I'm not really excited about documenting more precision than we are testing for...) > If someone comes along and complains because > oauth_server.py doesn't work on an older version, it's up to them > to provide a patch. As an aside, EL7's standard libcurl is too old to pass our current configure checks, so I think someone would really have to go out of their way to get to that point. --Jacob
Attachment
On 24.04.25 18:20, Jacob Champion wrote: > On Thu, Apr 24, 2025 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: >> I'm still content with the idea of deciding that 3.6 is now our >> cutoff. > > Seems like no one is pushing hard for an earlier version, yet, so > here's a patch with your suggested wording from upthread. I'm not sure > if this meets Peter's request for precision. (Though I'm not really > excited about documenting more precision than we are testing for...) I like the change to "supported", that's useful. I would just write 3.6 instead of 3.6.8. We've never tracked the third version component for Python.
Peter Eisentraut <peter@eisentraut.org> writes:
> On 24.04.25 18:20, Jacob Champion wrote:
>> Seems like no one is pushing hard for an earlier version, yet, so
>> here's a patch with your suggested wording from upthread. I'm not sure
>> if this meets Peter's request for precision. (Though I'm not really
>> excited about documenting more precision than we are testing for...)
> I like the change to "supported", that's useful.
> I would just write 3.6 instead of 3.6.8. We've never tracked the third
> version component for Python.
On the reading that "supported" means "we'll try to fix a problem
rather than telling you to use a newer Python", I suspect that the
correct thing to say is 3.6.8 not 3.6. There may be no difference
in practice; but if push comes to shove I don't think we'd support
a 3.6.x Python version that appears in no LTS distro.
regards, tom lane
On 29.04.25 17:16, Tom Lane wrote: > Peter Eisentraut <peter@eisentraut.org> writes: >> On 24.04.25 18:20, Jacob Champion wrote: >>> Seems like no one is pushing hard for an earlier version, yet, so >>> here's a patch with your suggested wording from upthread. I'm not sure >>> if this meets Peter's request for precision. (Though I'm not really >>> excited about documenting more precision than we are testing for...) > >> I like the change to "supported", that's useful. >> I would just write 3.6 instead of 3.6.8. We've never tracked the third >> version component for Python. > > On the reading that "supported" means "we'll try to fix a problem > rather than telling you to use a newer Python", I suspect that the > correct thing to say is 3.6.8 not 3.6. There may be no difference > in practice; but if push comes to shove I don't think we'd support > a 3.6.x Python version that appears in no LTS distro. Ok, that sounds sensible. FWIW, 3.6.8 is the "final bugfix release for 3.6", so at least it's not just some random intermediate version. https://www.python.org/downloads/release/python-3615/
On Tue, Apr 29, 2025 at 8:46 AM Peter Eisentraut <peter@eisentraut.org> wrote: > > On the reading that "supported" means "we'll try to fix a problem > > rather than telling you to use a newer Python", I suspect that the > > correct thing to say is 3.6.8 not 3.6. There may be no difference > > in practice; but if push comes to shove I don't think we'd support > > a 3.6.x Python version that appears in no LTS distro. > > Ok, that sounds sensible. Cool, I will plan to push this Sometime Soon, then. Thanks! --Jacob
On Tue, Apr 29, 2025 at 8:57 AM Jacob Champion <jacob.champion@enterprisedb.com> wrote: > Cool, I will plan to push this Sometime Soon, then. This is now committed. Thanks everybody! --Jacob