Thread: PATCH: Compiling PostgreSQL using ActiveState Python 3.2
Hi,
I am trying to build PostgreSQL 9.1beta3 using the ActiveState Python 3.2.
It did not compile successfully.
When I tried to figure out the exact reason for the failure, I found that:
1. 'python_configdir' variable is hardcoded, instead it should use the configuration 'LIBPL'.
2. 'plpython' is trying get linked using '-lpython${python_version}', but it should be '-lpython${python_ldversion}'.
Please find the attached patch, which resolve the issue on my side.
--
Thanks & Regards,
Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
Attachment
On ons, 2011-08-17 at 18:28 +0530, Ashesh Vashi wrote: > I am trying to build PostgreSQL 9.1beta3 using the ActiveState Python 3.2. > It did not compile successfully. Note that building against Python 3.2 works at least on Debian, so this is not a universal problem. It appears to have to do with the stable ABI thing they introduced in Python 3.2, so it will be mainly relevant to platforms targeted by that. > When I tried to figure out the exact reason for the failure, I found that: > 1. 'python_configdir' variable is hardcoded, instead it should use the > configuration 'LIBPL'. That looks reasonable. My Debian installation works around this by a symlink, but that's perhaps a hack they put in for this reason. > 2. 'plpython' is trying get linked using '-lpython${*python_version*}', but > it should be '-lpython${*python_ldversion*}'. That, on the other hand, will be a problem. get_config_vars('LDVERSION') isn't defined before Python 3.2, so this will break all previous versions. I find it a bit curious that this is necessary, because the previous coding works for me: $ python3.2 -c "import distutils.sysconfig,string; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY'))))" libpython3.2mu.so $ python3.2 -c "import distutils.sysconfig,string; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LDVERSION'))))" 3.2mu So it is not in fact true that we are linking against '-lpython ${*python_version*}'.
Peter Eisentraut <peter_e@gmx.net> writes: > On ons, 2011-08-17 at 18:28 +0530, Ashesh Vashi wrote: >> When I tried to figure out the exact reason for the failure, I found that: >> 1. 'python_configdir' variable is hardcoded, instead it should use the >> configuration 'LIBPL'. > That looks reasonable. My Debian installation works around this by a > symlink, but that's perhaps a hack they put in for this reason. FWIW, all three python installations I have handy (2.7 on Fedora 14, 2.7 on OS X Lion, 2.5 on HPUX) produce the same result from either of python -c "from distutils.sysconfig import get_python_lib as f; import os; print(os.path.join(f(plat_specific=1,standard_lib=1),'config'))" python -c "import distutils.sysconfig,string; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBPL'))))" It's not immediately apparent to me why we should think that get_python_lib is less trustworthy than LIBPL; but if someone can make that case, I don't have any objection to this part of the patch. >> 2. 'plpython' is trying get linked using '-lpython${*python_version*}', but >> it should be '-lpython${*python_ldversion*}'. > That, on the other hand, will be a problem. > get_config_vars('LDVERSION') isn't defined before Python 3.2, so this > will break all previous versions. Yes. In particular, this appears to be doing the wrong thing on my Lion installation: it changes python_libspec = -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 to just python_libspec = -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython and there is no libpython.dylib in that directory. The build accidentally fails to fail because there is a libpython.dylib in /usr/lib and it happens to be symlinked to the right version of python, but I hardly think we want to trust that. I'm also wondering why a patch that's supposed to enable building against python 3.2 should need to touch the "old way" code path. If 3.2 isn't using the "new way", what exactly does? regards, tom lane
On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote: > FWIW, all three python installations I have handy (2.7 on Fedora 14, 2.7 > on OS X Lion, 2.5 on HPUX) produce the same result from either of > > python -c "from distutils.sysconfig import get_python_lib as f; import os; print(os.path.join(f(plat_specific=1,standard_lib=1),'config'))" > python -c "import distutils.sysconfig,string; print(' '.join(filter(None,distutils.sysconfig.get_config_vars('LIBPL'))))" > > It's not immediately apparent to me why we should think that > get_python_lib is less trustworthy than LIBPL; but if someone > can make that case, I don't have any objection to this part of > the patch. The issue, at least for me, is that the file isn't necessarily called 'config' anymore. I have /usr/lib/python3.2/config-3.2mu because of some shared object ABI tagging system they introduced. (/usr/lib/python3.2/config is a symlink to that, as a transition measure, I guess.) LIBPL exists at least as far back as Python 2.2, so its use should be safe. > >> 2. 'plpython' is trying get linked using '-lpython${*python_version*}', but > >> it should be '-lpython${*python_ldversion*}'. > > > That, on the other hand, will be a problem. > > get_config_vars('LDVERSION') isn't defined before Python 3.2, so this > > will break all previous versions. > > Yes. In particular, this appears to be doing the wrong thing on my Lion > installation: it changes > python_libspec = -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 > to just > python_libspec = -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython > and there is no libpython.dylib in that directory. The build > accidentally fails to fail because there is a libpython.dylib > in /usr/lib and it happens to be symlinked to the right version of > python, but I hardly think we want to trust that. Yes, because get_config_vars('LDVERSION') doesn't exist in that version. In theory, it would return '2.7', so everything would fit back together, but LDVERSION doesn't exist before 3.2. > I'm also wondering why a patch that's supposed to enable building > against python 3.2 should need to touch the "old way" code path. > If 3.2 isn't using the "new way", what exactly does? Analogously to the point above, the result on my system should be -L something -lpython3.2mu And that's what I get. The claim is that on the ActiveState installation, this doesn't work out, but we need to see some details here, I guess.
Peter Eisentraut <peter_e@gmx.net> writes: > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote: >> It's not immediately apparent to me why we should think that >> get_python_lib is less trustworthy than LIBPL; but if someone >> can make that case, I don't have any objection to this part of >> the patch. > The issue, at least for me, is that the file isn't necessarily called > 'config' anymore. I have > /usr/lib/python3.2/config-3.2mu Ah, I see. > LIBPL exists at least as far back as Python 2.2, so its use should be > safe. Yeah, that part of the patch seems sane then. > Yes, because get_config_vars('LDVERSION') doesn't exist in that version. > In theory, it would return '2.7', so everything would fit back together, > but LDVERSION doesn't exist before 3.2. Could we have the code use 'LDVERSION' if it gets a nonempty result, and otherwise fall back to the current scheme? But I guess first we need some details as to why the current scheme isn't sufficient. regards, tom lane
![](http://s.wisestamp.com/pixel.png?p=mozilla&v=2.0.3&t=1285751372076&u=4060523&e=777)
On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
--
Peter Eisentraut <peter_e@gmx.net> writes:> On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:>> It's not immediately apparent to me why we should think that
>> get_python_lib is less trustworthy than LIBPL; but if someone
>> can make that case, I don't have any objection to this part of
>> the patch.
> The issue, at least for me, is that the file isn't necessarily called
> 'config' anymore. I have
> /usr/lib/python3.2/config-3.2mu
One of the reason, I say that - we do have hard-coded values for the config directory.
Hence, I used the LIBPL.
Ah, I see.Yeah, that part of the patch seems sane then.
> LIBPL exists at least as far back as Python 2.2, so its use should be
> safe.
> Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
> In theory, it would return '2.7', so everything would fit back together,
> but LDVERSION doesn't exist before 3.2.
Oops - sorry...
I did not know about it..
Could we have the code use 'LDVERSION' if it gets a nonempty result,
and otherwise fall back to the current scheme? But I guess first we
need some details as to why the current scheme isn't sufficient.
Please find the attached patch as you suggested.
Reason:
- As per my findings, ActiveState Python 3.2 does not provide shared libraries along with it.
(Though - I am not sure about the earlier version of ActiveState Python)
We can confirm the same using the following command:
${PYTHON} -c "import distutils.sysconfig,string; print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
Which returns in this case '0'.
And, getting values for the 'python_ldlibrary' and 'python_so' are 'libpython3.2m.a' and '.cpython-32m.so' respectively.
So, the condition - x"${python_ldlibrary}" != x"${ldlibrary}" is always failing, and switching it back to link the old way.
--
Thanks & Regards,
Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
regards, tom lane
Attachment
Please ignore the previous patch.
--
![](http://s.wisestamp.com/pixel.png?p=mozilla&v=2.0.3&t=1285751372076&u=4060523&e=777)
Please find the updated patch.
--
Thanks & Regards,
Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
![](http://s.wisestamp.com/pixel.png?p=mozilla&v=2.0.3&t=1285751372076&u=4060523&e=777)
On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:Peter Eisentraut <peter_e@gmx.net> writes:> On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:>> It's not immediately apparent to me why we should think that
>> get_python_lib is less trustworthy than LIBPL; but if someone
>> can make that case, I don't have any objection to this part of
>> the patch.
> The issue, at least for me, is that the file isn't necessarily called
> 'config' anymore. I have
> /usr/lib/python3.2/config-3.2muOne of the reason, I say that - we do have hard-coded values for the config directory.Hence, I used the LIBPL.Ah, I see.Yeah, that part of the patch seems sane then.
> LIBPL exists at least as far back as Python 2.2, so its use should be
> safe.
> Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
> In theory, it would return '2.7', so everything would fit back together,
> but LDVERSION doesn't exist before 3.2.Oops - sorry...I did not know about it..Could we have the code use 'LDVERSION' if it gets a nonempty result,
and otherwise fall back to the current scheme? But I guess first we
need some details as to why the current scheme isn't sufficient.Please find the attached patch as you suggested.Reason:- As per my findings, ActiveState Python 3.2 does not provide shared libraries along with it.(Though - I am not sure about the earlier version of ActiveState Python)We can confirm the same using the following command:${PYTHON} -c "import distutils.sysconfig,string; print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"Which returns in this case '0'.And, getting values for the 'python_ldlibrary' and 'python_so' are 'libpython3.2m.a' and '.cpython-32m.so' respectively.So, the condition - x"${python_ldlibrary}" != x"${ldlibrary}" is always failing, and switching it back to link the old way.
regards, tom lane
Attachment
On tor, 2011-08-18 at 14:51 +0530, Ashesh Vashi wrote: > Please ignore the previous patch. > Please find the updated patch. Committed more or less like that. In passing I also fixed the build with Python 3 on Windows, which apparently never worked before. But I suppose you have been referring to the ActiveState Linux build all along. > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company<http://www.enterprisedb.com> > > > > *http://www.linkedin.com/in/asheshvashi*<http://www.linkedin.com/in/asheshvashi> > > > > On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi < > ashesh.vashi@enterprisedb.com> wrote: > > > On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > > >> Peter Eisentraut <peter_e@gmx.net> writes: > >> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote: > >> >> It's not immediately apparent to me why we should think that > >> >> get_python_lib is less trustworthy than LIBPL; but if someone > >> >> can make that case, I don't have any objection to this part of > >> >> the patch. > >> > >> > The issue, at least for me, is that the file isn't necessarily called > >> > 'config' anymore. I have > >> > /usr/lib/python3.2/config-3.2mu > >> > > One of the reason, I say that - we do have hard-coded values for the config > > directory. > > Hence, I used the LIBPL. > > > >> > >> Ah, I see. > >> > >> > LIBPL exists at least as far back as Python 2.2, so its use should be > >> > safe. > >> > >> Yeah, that part of the patch seems sane then. > >> > >> > Yes, because get_config_vars('LDVERSION') doesn't exist in that version. > >> > In theory, it would return '2.7', so everything would fit back together, > >> > but LDVERSION doesn't exist before 3.2. > >> > > Oops - sorry... > > I did not know about it.. > > > >> > >> Could we have the code use 'LDVERSION' if it gets a nonempty result, > >> and otherwise fall back to the current scheme? But I guess first we > >> need some details as to why the current scheme isn't sufficient. > >> > > Please find the attached patch as you suggested. > > > > Reason: > > - As per my findings, ActiveState Python 3.2 does not provide shared > > libraries along with it. > > (Though - I am not sure about the earlier version of ActiveState Python) > > We can confirm the same using the following command: > > ${PYTHON} -c "import distutils.sysconfig,string; > > print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))" > > Which returns in this case '0'. > > > > And, getting values for the 'python_ldlibrary' and 'python_so' are > > 'libpython3.2m.a' and '.cpython-32m.so' respectively. > > So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is always > > failing, and switching it back to link the old way. > > > > -- > > > > Thanks & Regards, > > > > Ashesh Vashi > > EnterpriseDB INDIA: Enterprise PostgreSQL Company<http://www.enterprisedb.com/> > > > > > > > > *http://www.linkedin.com/in/asheshvashi*<http://www.linkedin.com/in/asheshvashi> > > > > > >> > >> regards, tom lane > >> > > > >
![](http://s.wisestamp.com/pixel.png?p=mozilla&v=2.0.3&t=1285751372076&u=4060523&e=777)
On Thu, Aug 18, 2011 at 5:25 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On tor, 2011-08-18 at 14:51 +0530, Ashesh Vashi wrote:Committed more or less like that.
> Please ignore the previous patch.
> Please find the updated patch.
Thanks
Yes.
In passing I also fixed the build with Python 3 on Windows, which
apparently never worked before. But I suppose you have been referring
to the ActiveState Linux build all along.
--
Thanks & Regards,
Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
>
> --
>
> Thanks & Regards,
>
> Ashesh Vashi
> EnterpriseDB INDIA: Enterprise PostgreSQL Company<http://www.enterprisedb.com>
>
>
>
> *http://www.linkedin.com/in/asheshvashi*<http://www.linkedin.com/in/asheshvashi>> > EnterpriseDB INDIA: Enterprise PostgreSQL Company<http://www.enterprisedb.com/>>
>
>
> On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi <
> ashesh.vashi@enterprisedb.com> wrote:
>
> > On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> >> Peter Eisentraut <peter_e@gmx.net> writes:
> >> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:
> >> >> It's not immediately apparent to me why we should think that
> >> >> get_python_lib is less trustworthy than LIBPL; but if someone
> >> >> can make that case, I don't have any objection to this part of
> >> >> the patch.
> >>
> >> > The issue, at least for me, is that the file isn't necessarily called
> >> > 'config' anymore. I have
> >> > /usr/lib/python3.2/config-3.2mu
> >>
> > One of the reason, I say that - we do have hard-coded values for the config
> > directory.
> > Hence, I used the LIBPL.
> >
> >>
> >> Ah, I see.
> >>
> >> > LIBPL exists at least as far back as Python 2.2, so its use should be
> >> > safe.
> >>
> >> Yeah, that part of the patch seems sane then.
> >>
> >> > Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
> >> > In theory, it would return '2.7', so everything would fit back together,
> >> > but LDVERSION doesn't exist before 3.2.
> >>
> > Oops - sorry...
> > I did not know about it..
> >
> >>
> >> Could we have the code use 'LDVERSION' if it gets a nonempty result,
> >> and otherwise fall back to the current scheme? But I guess first we
> >> need some details as to why the current scheme isn't sufficient.
> >>
> > Please find the attached patch as you suggested.
> >
> > Reason:
> > - As per my findings, ActiveState Python 3.2 does not provide shared
> > libraries along with it.
> > (Though - I am not sure about the earlier version of ActiveState Python)
> > We can confirm the same using the following command:
> > ${PYTHON} -c "import distutils.sysconfig,string;
> > print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
> > Which returns in this case '0'.
> >
> > And, getting values for the 'python_ldlibrary' and 'python_so' are
> > 'libpython3.2m.a' and '.cpython-32m.so' respectively.
> > So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is always
> > failing, and switching it back to link the old way.
> >
> > --
> >
> > Thanks & Regards,
> >
> > Ashesh Vashi
> >
> >
> >
> > *http://www.linkedin.com/in/asheshvashi*<http://www.linkedin.com/in/asheshvashi>
> >
> >
> >>
> >> regards, tom lane
> >>
> >
> >