Thread: unable to build postgres-9.4 in os x 10.9 with python

Hi,

has anyone else had this issue and does any one know the solution?

Since upgrading my mac from os x 10.8 to 10.9, i can no long build postgres with '--with-python’.

i get the following error.

ld: framework not found Python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [plpython2.so] Error 1
make[2]: *** [all-plpython-recurse] Error 2
make[1]: *** [all-pl-recurse] Error 2
make: *** [all-src-recurse] Error 2

my full configure command is

configure '--prefix=/usr/local/postgres/9.4' '--with-perl' '--with-python' '--with-libxml' '-with-openssl'
'--with-includes=/opt/local/include/''--with-libraries=/opt/local/lib' 

any help would be appreciated.

reiner

Hi,

has anyone else had this issue and does any one know the solution?

Since upgrading my mac from os x 10.8 to 10.9, i can no long build postgres with '--with-python’.

i get the following error.

ld: framework not found Python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [plpython2.so] Error 1
make[2]: *** [all-plpython-recurse] Error 2
make[1]: *** [all-pl-recurse] Error 2
make: *** [all-src-recurse] Error 2

my full configure command is

configure '--prefix=/usr/local/postgres/9.4' '--with-perl' '--with-python' '--with-libxml' '-with-openssl' '--with-includes=/opt/local/include/' '--with-libraries=/opt/local/lib'

Asif Naeem <anaeem.it@gmail.com> writes:
> On Fri, May 30, 2014 at 9:49 PM, reiner peterke <zedaardv@drizzle.com>
> wrote:
>> Since upgrading my mac from os x 10.8 to 10.9, i can no long build
>> postgres with '--with-python’.

> Latest PG 9.4 source code seems building fine with --with-python option on
> my OS X 10.9.3 box  i.e.

[ please don't top-post in PG mailing lists ]

I can reproduce the failure as described, not only in 9.4 but the back
branches too; I would've noticed sooner except I don't build plpython
routinely.

It looks to me like Apple broke something in the build toolchain.
If you add -v as suggested, what you see is

$ make PROFILE="-v"
gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -g -v  -bundle -multiply_defined suppress -o
plpython2.soplpy_cursorobject.o plpy_elog.o plpy_exec.o plpy_main.o plpy_planobject.o plpy_plpymodule.o
plpy_procedure.oplpy_resultobject.o plpy_spi.o plpy_subxactobject.o plpy_typeio.o plpy_util.o -L../../../src/port
-L../../../src/common-Wl,-dead_strip_dylibs  -v  -F/System/Library/Frameworks -framework Python   -bundle_loader
../../../src/backend/postgres
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.2.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -dynamic -arch
x86_64-bundle -bundle_loader ../../../src/backend/postgres -macosx_version_min 10.9.0 -multiply_defined suppress
-syslibroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -o
plpython2.so-L../../../src/port -L../../../src/common plpy_cursorobject.o plpy_elog.o plpy_exec.o plpy_main.o
plpy_planobject.oplpy_plpymodule.o plpy_procedure.o plpy_resultobject.o plpy_spi.o plpy_subxactobject.o plpy_typeio.o
plpy_util.o-dead_strip_dylibs -framework Python -lSystem
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin/libclang_rt.osx.a
-F/System/Library/Frameworks
ld: framework not found Python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [plpython2.so] Error 1

The problem is that that -syslibroot option modifies where to look for
frameworks; "man ld" quoth

     -syslibroot rootdir
                 Prepend rootdir to all search paths when searching for
                 libraries or frameworks.

If you do the ld call by hand without the -syslibroot option, it works.
AFAICS it could never have worked with such an option, so I'm thinking
this is some new misbehavior in the latest version of Xcode.

            regards, tom lane

On 05/30/2014 11:52 AM, Tom Lane wrote:
> Asif Naeem <anaeem.it@gmail.com> writes:
>> On Fri, May 30, 2014 at 9:49 PM, reiner peterke <zedaardv@drizzle.com>
>> wrote:
>>> Since upgrading my mac from os x 10.8 to 10.9, i can no long build
>>> postgres with '--with-python’.
>
>> Latest PG 9.4 source code seems building fine with --with-python option on
>> my OS X 10.9.3 box  i.e.
>
> [ please don't top-post in PG mailing lists ]
>
> I can reproduce the failure as described, not only in 9.4 but the back
> branches too; I would've noticed sooner except I don't build plpython
> routinely.
>
> It looks to me like Apple broke something in the build toolchain.
> If you add -v as suggested, what you see is

> If you do the ld call by hand without the -syslibroot option, it works.
> AFAICS it could never have worked with such an option, so I'm thinking
> this is some new misbehavior in the latest version of Xcode.

There is and the SO thread that goes into detail on this is here:

http://stackoverflow.com/questions/19555395/python-framework-is-missing-from-os-x-10-9-sdk-why-also-workaround

The Apple document referenced in above is:

https://developer.apple.com/library/ios/technotes/tn2328/_index.html

>
>             regards, tom lane
>
>


--
Adrian Klaver
adrian.klaver@aklaver.com

Adrian Klaver <adrian.klaver@aklaver.com> writes:
> On 05/30/2014 11:52 AM, Tom Lane wrote:
>> If you do the ld call by hand without the -syslibroot option, it works.
>> AFAICS it could never have worked with such an option, so I'm thinking
>> this is some new misbehavior in the latest version of Xcode.

> There is and the SO thread that goes into detail on this is here:
> http://stackoverflow.com/questions/19555395/python-framework-is-missing-from-os-x-10-9-sdk-why-also-workaround
> The Apple document referenced in above is:
> https://developer.apple.com/library/ios/technotes/tn2328/_index.html

Fun.  So after all these years of catering to Apple's preferred weirdness
in this regard, they reverse course and tell us to do it like everywhere
else.

I experimented with just diking out the python_framework case in
configure, and that *almost* works; but for some reason distutils doesn't
admit to having a shared library, so you also have to override that test.

The attached patch fixes this and also removes a long-obsolete comment
claiming that we don't work with Python < 2.5 on OSX (see prairiedog
for evidence to the contrary).

I've tested this successfully on my 10.9.3 laptop as well as on dromedary
and prairiedog, so I'm thinking we should patch not only HEAD but all the
back branches.  Any objections?

            regards, tom lane

PS: why aren't any of the buildfarm members using 10.9 building
--with-python?  We should have known about this months ago, ISTM.

diff --git a/config/python.m4 b/config/python.m4
index 5cb2854..7012c53 100644
*** a/config/python.m4
--- b/config/python.m4
*************** python_libdir=`${PYTHON} -c "import dist
*** 68,81 ****
  python_ldlibrary=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY'))))"`
  python_so=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('SO'))))"`
  ldlibrary=`echo "${python_ldlibrary}" | sed "s/${python_so}$//"`
- python_framework=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('PYTHONFRAMEWORK'))))"`
  python_enable_shared=`${PYTHON} -c "import distutils.sysconfig;
print(distutils.sysconfig.get_config_vars().get('Py_ENABLE_SHARED',0))"`

! if test -n "$python_framework"; then
!     python_frameworkprefix=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('PYTHONFRAMEWORKPREFIX'))))"`
!     python_libspec="-F${python_frameworkprefix} -framework $python_framework"
!     python_enable_shared=1
! elif test x"${python_libdir}" != x"" -a x"${python_ldlibrary}" != x"" -a x"${python_ldlibrary}" != x"${ldlibrary}"
  then
      # New way: use the official shared library
      ldlibrary=`echo "${ldlibrary}" | sed "s/^lib//"`
--- 68,76 ----
  python_ldlibrary=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY'))))"`
  python_so=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('SO'))))"`
  ldlibrary=`echo "${python_ldlibrary}" | sed "s/${python_so}$//"`
  python_enable_shared=`${PYTHON} -c "import distutils.sysconfig;
print(distutils.sysconfig.get_config_vars().get('Py_ENABLE_SHARED',0))"`

! if test x"${python_libdir}" != x"" -a x"${python_ldlibrary}" != x"" -a x"${python_ldlibrary}" != x"${ldlibrary}"
  then
      # New way: use the official shared library
      ldlibrary=`echo "${ldlibrary}" | sed "s/^lib//"`
*************** else
*** 91,99 ****
      python_libspec="-L${python_libdir} -lpython${python_ldversion}"
  fi

! if test -z "$python_framework"; then
!     python_additional_libs=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`
! fi

  AC_MSG_RESULT([${python_libspec} ${python_additional_libs}])

--- 86,92 ----
      python_libspec="-L${python_libdir} -lpython${python_ldversion}"
  fi

! python_additional_libs=`${PYTHON} -c "import distutils.sysconfig; print('
'.join(filter(None,distutils.sysconfig.get_config_vars('LIBS','LIBC','LIBM','BASEMODLIBS'))))"`

  AC_MSG_RESULT([${python_libspec} ${python_additional_libs}])

diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 46d2030..020861a 100644
*** a/src/pl/plpython/Makefile
--- b/src/pl/plpython/Makefile
*************** include $(top_builddir)/src/Makefile.glo
*** 9,24 ****
  # asks Python directly.  But because this has been broken in Debian
  # for a long time (http://bugs.debian.org/695979), and to support
  # older Python versions, we see if there is a file that is named like
! # a shared library as a fallback.  (Note that this is wrong on OS X,
! # where DLSUFFIX is .so, but libpython is a .dylib.  Python <2.5 is
! # therefore not supported on OS X.)
  ifeq (1,$(python_enable_shared))
  shared_libpython = yes
  else
  ifneq (,$(wildcard $(python_libdir)/libpython*$(DLSUFFIX)*))
  shared_libpython = yes
  endif
  endif

  # Windows needs to convert backslashed paths to normal slashes,
  # and we have to remove -lpython from the link since we are building our own
--- 9,27 ----
  # asks Python directly.  But because this has been broken in Debian
  # for a long time (http://bugs.debian.org/695979), and to support
  # older Python versions, we see if there is a file that is named like
! # a shared library as a fallback.
  ifeq (1,$(python_enable_shared))
  shared_libpython = yes
  else
+ ifeq ($(PORTNAME), darwin)
+ # OS X does supply a .dylib even though Py_ENABLE_SHARED does not get set
+ shared_libpython = yes
+ else
  ifneq (,$(wildcard $(python_libdir)/libpython*$(DLSUFFIX)*))
  shared_libpython = yes
  endif
  endif
+ endif

  # Windows needs to convert backslashed paths to normal slashes,
  # and we have to remove -lpython from the link since we are building our own

On 05/30/2014 01:31 PM, Tom Lane wrote:
> Adrian Klaver <adrian.klaver@aklaver.com> writes:
>> On 05/30/2014 11:52 AM, Tom Lane wrote:
>>> If you do the ld call by hand without the -syslibroot option, it works.
>>> AFAICS it could never have worked with such an option, so I'm thinking
>>> this is some new misbehavior in the latest version of Xcode.
>
>> There is and the SO thread that goes into detail on this is here:
>> http://stackoverflow.com/questions/19555395/python-framework-is-missing-from-os-x-10-9-sdk-why-also-workaround
>> The Apple document referenced in above is:
>> https://developer.apple.com/library/ios/technotes/tn2328/_index.html
>
> Fun.  So after all these years of catering to Apple's preferred weirdness
> in this regard, they reverse course and tell us to do it like everywhere
> else.

Mavericks, the gift that keeps giving:)



--
Adrian Klaver
adrian.klaver@aklaver.com

Hi all,

Thanks for all the responses.

It looks like Edoardo was on the right track,
I hadn’t upgraded the Command Line Developer Tools.
after downloading them from apple and installing them, i was able to build postgres with '--with-python'


reiner

On 30 maj 2014, at 19:56, Edoardo Panfili <edoardo@aspix.it> wrote:

> Il 30/05/14 18:49, reiner peterke ha scritto:
>> Hi,
>>
>> has anyone else had this issue and does any one know the solution?
>>
>> Since upgrading my mac from os x 10.8 to 10.9, i can no long build postgres with '--with-python’.
>>
>> i get the following error.
>>
>> ld: framework not found Python
>> clang: error: linker command failed with exit code 1 (use -v to see invocation)
>> make[3]: *** [plpython2.so] Error 1
>> make[2]: *** [all-plpython-recurse] Error 2
>> make[1]: *** [all-pl-recurse] Error 2
>> make: *** [all-src-recurse] Error 2
>>
>> my full configure command is
>>
>> configure '--prefix=/usr/local/postgres/9.4' '--with-perl' '--with-python' '--with-libxml' '-with-openssl'
'--with-includes=/opt/local/include/''--with-libraries=/opt/local/lib' 
> I did I try with 9.4beta1 and --with-python using OS X 10.9.3
>
> ./configure --with-python
>
> all goes well, have you installed the command line tools?
>
> xcode-select --install
>
> I did it to compile uuid (with no success) and postgis (with success).
>
> Edoardo