Thread: "Fat" binaries for OS X (was Re: [GENERAL] Postgres Library natively available for Mac OSX Intel?)

Philipp Ott <philipp.ott@avalon.at> writes:
> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode  
> 2.2.1, but generates binaries just for the current host architecture.  
> Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for  
> configure, then it compiles everything just fine, however at linking  
> stage I get various problems for missing architecture files.

I looked into this and found that the problem is our habit of using
"ld -r" to aggregate multiple .o files into a single SUBSYS.o file
that's still relocatable.  The Darwin version of ld is pretty brain-dead
when it comes to "fat" files containing code for more than one
architecture --- the man page says

UNIVERSAL FILE SUPPORT      The link editor  accepts  ``universal''  (multiple-architecture)  input      files,  but
always creates a ``thin'' (single-architecture), standard      Mach-O output file.  The architecture is specified
using the  -arch      arch_type option.  If this option is not used, ld(1) attempts to deter-      mine the output
architectureby examining the first object file encoun-      tered  on the command line.  If it is a ``thin'' file, its
architecture     determines that of the output file.  If  the  first  input  file  is  a      ``universal''  file,  the
``best''  architecture for the host is used.      (See the explanation of the -arch option, below.)
 
      The compiler driver cc(1) handles  creating  universal  executables  by      calling  ld(1)  multiple  times and
usinglipo(1) to create a ``univer-      sal'' file from the results of the ld(1) executions.
 

So what you're seeing is that one of the arches has been dropped from
the SUBSYS.o files.

Unfortunately, there doesn't seem to be any way to use cc/gcc to emulate
"ld -r", in the sense of just combining multiple fat .o files into one
fat .o file.  At least I couldn't see one after perusing the man page
for a bit.  I also found out that lipo(1) is not by itself smart enough
to do this.

So it looks like you'd have to write a small shell script to do what the
above snippet describes cc as doing.  Not out of the question by any
means, but still a PITA.  Any Apple experts around who know a better
answer?  Is Apple likely to improve this situation in the near future?

BTW, our configure script is not real flexible about adjusting the
command used to produce the SUBSYS.o files ... if you want anything
except "$(LD) -r -o SUBSYS.o *.o", you have to edit Makefile.global
after configuring.  But without having a solution that actually works
for multi-arch Darwin, I'm not seeing the point of improving that yet.
        regards, tom lane

Hi Tom!


Am 09.04.2006 um 04:34 schrieb Tom Lane:

> Philipp Ott <philipp.ott@avalon.at> writes:
>> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode
>> 2.2.1, but generates binaries just for the current host architecture.
>> Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for
>> configure, then it compiles everything just fine, however at linking
>> stage I get various problems for missing architecture files.
>
> [...]
> So what you're seeing is that one of the arches has been dropped from
> the SUBSYS.o files.
>
> Unfortunately, there doesn't seem to be any way to use cc/gcc to  
> emulate
> "ld -r", in the sense of just combining multiple fat .o files into one
> fat .o file.  At least I couldn't see one after perusing the man page
> for a bit.  I also found out that lipo(1) is not by itself smart  
> enough
> to do this.
>
> So it looks like you'd have to write a small shell script to do  
> what the
> above snippet describes cc as doing.  Not out of the question by any
> means, but still a PITA.  Any Apple experts around who know a better
> answer?  Is Apple likely to improve this situation in the near future?
> [...]


Thank you for the suggestion. I followed your suggestion and came up  
with a ldfat script and a change to src/Makefile.global after running  
configure. I put my ldfat script into the top_builddir and reference  
it in the src/Makefile.global.

So from the download of postgresql-8.1.3.tar.gz I made the following  
steps:

tar xzf postgresql-8.1.3.tar.gz
cd postgres-8.1.3
./configure CFLAGS="-arch i386 -arch ppc"

vi src/Makefile.global
....
LD = ${top_builddir}/ldfat
...


And in the postgres top directory I put the following script called  
ldfat.


#!/bin/bash

OFILES=""
RELOPT=""
OUTPUT=""
OTHERS=""

while [ "$#" != "0" ];
do  case "$1" in    -r) RELOPT="-r";;    -o) OUTPUT=`basename -s .o "$2"`; shift;;    *.o) OFILES="$OFILES $1";;    *)
OTHERS="$OTHERS$1";;  esac  shift
 
done

if [ "$RELOPT" == "-r" ];
then  echo ldfat $RELOPT -o $OUTPUT $OFILES $OTHERS  `/usr/bin/ld -r -arch i386 -o ${OUTPUT}_i386.o $OFILES $OTHERS`
`/usr/bin/ld-r -arch ppc -o ${OUTPUT}_ppc.o $OFILES $OTHERS`  `lipo -create -output ${OUTPUT}.o ${OUTPUT}_i386.o
${OUTPUT}_ppc.o`
else  echo ld -o $OUTPUT $OFILES $OTHERS  `/usr/bin/ld -o $OUTPUT $OFILES $OTHERS`
fi
exit $?


Now make and sudo make install run fine and in default config put a  
working psql (and the rest I think will work too) into /usr/local/ 
pgsql on my imac. At the moment I just have this Intel iMac around  
but I will try this with a PowerPC Mac later and let you know, if the  
generated binaries and libraries work on an out-of-the-box OSX.

I vote for an --enable-osxuniversal option to ./configure which adds  
"-arch ppc -arch i386" to the CFLAGS and replaces "LD" with a script  
solution for the time being, unless an Apple expert has a better  
solution at hand. If Apple makes changes to ld later this --enable- 
osxuniversal would handle this transparently. However my configure  
knowledge is next to non-existant :-) Also I only made a cursory  
glance over all Makefiles ${LD} invoctions for calling patterns and  
atm it works, however it would need some checking for available  
options and errors.

Regards,
Philipp Ott