Thread: Re: [GENERAL] Can't Build 7.3.4 on OS X

Hunter Hillegas <lists@lastonepicked.com> writes:
> I cannot build the latest release on OS X Jaguar.
> Running GCC 3.3 from Apple:

It seems "-traditional-cpp" has become nontraditional in 3.3.  Or
possibly Apple changed their system header files in a way that broke
that preprocessor.  What's certain is that gcc 3.3 rejects some of
the Darwin header files when using -traditional-cpp.  This is not our
fault ;-).

It seems to work okay if you change -traditional-cpp to -no-cpp-precomp
in src/template/darwin (you will need to rerun configure afterwards).

Experimenting, I find that Apple's gcc 3.1 and 3.3 both build PG fine
with that switch.  Their latest update of gcc 2.95 no longer seems to
work at all (it generates invalid assembler code for xlog.c).

I tried removing the cpp option entirely, but that blew up in other
places, and I have no interest in figuring out why just now.

Is anyone on the list still running OS X 10.1, or anyway still using a
version of the OS X developer tools older than the Dec 2002 release?
It would be good to check if -no-cpp-precomp creates any problems on
any release that anyone still cares about.  For the moment, I've made
src/template/darwin unconditionally use -no-cpp-precomp, but we could
probably hack it to use -traditional-cpp if there's still any interest
in Darwin compiler versions that don't recognize -no-cpp-precomp.

            regards, tom lane

On lauantai, syys 20, 2003, at 23:37 Europe/Helsinki, Tom Lane wrote:
> Is anyone on the list still running OS X 10.1, or anyway still using a
> version of the OS X developer tools older than the Dec 2002 release?
> It would be good to check if -no-cpp-precomp creates any problems on
> any release that anyone still cares about.  For the moment, I've made
> src/template/darwin unconditionally use -no-cpp-precomp, but we could
> probably hack it to use -traditional-cpp if there's still any interest
> in Darwin compiler versions that don't recognize -no-cpp-precomp.

-no-cpp-precomp replaced -traditional-cpp as the preferred syntax
for turning off the precompiled header support in Mac OS X 10.1
(Darwin 5.0). Supporting Mac OS X 10.0 (Darwin 1.x) would require
-traditional-cpp, but I don't think it's necessary. 10.1 was a free
upgrade and it didn't remove any hardware support, so I really
can't see a reason why somebody would continue to use it.

mk

Marko Karppinen <karppinen@pobox.com> writes:
> On lauantai, syys 20, 2003, at 23:37 Europe/Helsinki, Tom Lane wrote:
>> Is anyone on the list still running OS X 10.1, or anyway still using a
>> version of the OS X developer tools older than the Dec 2002 release?

> -no-cpp-precomp replaced -traditional-cpp as the preferred syntax
> for turning off the precompiled header support in Mac OS X 10.1
> (Darwin 5.0).

Great.  I was afraid it might have been new with 10.2.

> Supporting Mac OS X 10.0 (Darwin 1.x) would require
> -traditional-cpp, but I don't think it's necessary.

I agree.  Thanks for the info.

BTW, is anyone interested in looking into whether we can be made to
build without using either flag?  I tried it and saw a number of
failures that looked like they traced to incompatible macro expansion.
This wouldn't surprise me if PG were some halfbaked package that only
got tested with stock GCC, but considering that we actually build fine
on quite a range of compilers, it seems like Apple's cpp ought to work
too.  It would be interesting to understand what the problem is.

            regards, tom lane

Tom:
> BTW, is anyone interested in looking into whether we can be made to
> build without using either flag?  I tried it and saw a number of
> failures that looked like they traced to incompatible macro expansion.
> This wouldn't surprise me if PG were some halfbaked package that only
> got tested with stock GCC, but considering that we actually build fine
> on quite a range of compilers, it seems like Apple's cpp ought to work
> too.  It would be interesting to understand what the problem is.

Does this happen with gcc 3.3? My understanding is that pre-3.3, the  
default C preprocessor was cpp-precomp, a NeXT-era concoction that  
supports precompiled header files for Objective-C. This preprocessor is  
about as bad as it seems to be, so even Apple recommends turning it off  
for other languages.

In gcc 3.3, Apple has generalized the precompiled headers feature to a  
mainline gcc feature called PFE, so cpp-precomp should be history. If  
I've gotten this right, you shouldn't get (the same) compiler errors  
with gcc 3.3 if you leave out the -no-cpp-precomp switch.

. . .

While we're on a Mac-related note, I managed to compile PostgreSQL on  
Mac OS X 10.2 Jaguar with two-level namespace support.  
<http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/ 
TwoLevelNamespaces.html> This is quite useful, so I hope you can look  
into incorporating the changes.

http://www.markokarppinen.com/pg-two-level-namespace.diff

This involves getting rid of undefined symbols in all shared libraries  
(or "bundles") that get built. I tested this with "./configure" and  
"./configure --with-tcl" but didn't try the jdbc or python extensions.  
Hopefully someone else can test those, and if all seems fine, commit  
the changes.

Thanks,
Marko

On Sep 21, 2003, at 3:11 PM, Tom Lane wrote:

> Great.  I was afraid it might have been new with 10.2.

Also, 7.3.4 doesn't link on the OS X 10.3 beta's.  Apparently tas is
never being defined.  I could never fix this.  In the list archives I
found all sorts of references to tas()/TAS and older versions of
postgres on other operating systems, but I simply couldn't figure out
how to make it happen on OS X 10.3.

the #if defined(__APPLE__) || defined(__ppc__)  section of s_lock.c was
being used, as best I could tell.  It defines a method named "void
tas_dummy()".  And s_lock.h declares "extern int tas(slock_t *lock)"
and "#define TAS(lock)  tas(lock)".... but how those match up to
tas_dummy() I don't know.

Everything compiles, then ld says:

ld: Undefined Symbol
_tas

any ideas here?  7.3.2 and 7.4beta3 compile just fine (I noticed that
7.4 has something more cross-platform for tas).  What happened in 7.3.4
that broke it?

eric

On Sep 21, 2003, at 3:11 PM, Tom Lane wrote:

> BTW, is anyone interested in looking into whether we can be made to
> build without using either flag?  I tried it and saw a number of

I did this... before I knew about -no-cpp-precomp.   :(  I read all
about -traditional-cpp in the gcc man page, but could never find the
corresponding "not traditional cpp" flag.

It boiled down to two things:  use of macros that used the
"stringification" syntax, and whitespace around marco arguments.

Take src/include/nodes/nodes.h, around line 265 for example:

#define makeNode(_type_)  ((_type_ *) newNode(sizeof(_type_),T_#_type_))
...
#define IsA(nodeptr, _type_) (nodeTag(nodeptr) == T_#_type_)

gcc 3.3 just didn't like this.  So I had to fake it out:

#define T_UNDER()  T_
#define makeNode(_type_)  ((_type_ *)
newNode(sizeof(_type_),T_UNDER()_type_))
...
#define IsA(nodeptr,_type_)     (nodeTag(nodeptr) == T_UNDER()_type_)

But it gets better.   Apparently with gcc 3.3 whitespace around macro
arguments is preserved!  So, in the case of calls to (at least) the IsA
macro:

before:  if (IsA(foo, Short))
after:      if (IsA(foo,Short))
                                 ^----------------- no space!

 From what I could tell, the statement would be expanded into (using my
re-defined version above):
    if (nodeTag(nodeptr) == T_ Short)

which of course isn't legal syntax b/c of the space.

So I went through with some Perl and did a bunch of global
substitutions on the files that gcc complained about.  There were a few
more than the above examples, but not too many.

> too.  It would be interesting to understand what the problem is.

There it is.

eric

Eric Ridge <ebr@tcdi.com> writes:
> any ideas here?  7.3.2 and 7.4beta3 compile just fine (I noticed that
> 7.4 has something more cross-platform for tas).  What happened in 7.3.4
> that broke it?

That makes no sense at all --- AFAICT there were *no* darwin or ppc
specific changes between 7.3.2 and 7.3.4.  Can you double check?

            regards, tom lane

On Sep 21, 2003, at 9:03 PM, Tom Lane wrote:

> Eric Ridge <ebr@tcdi.com> writes:
>> any ideas here?  7.3.2 and 7.4beta3 compile just fine (I noticed that
>> 7.4 has something more cross-platform for tas).  What happened in
>> 7.3.4
>> that broke it?
>
> That makes no sense at all --- AFAICT there were *no* darwin or ppc
> specific changes between 7.3.2 and 7.3.4.  Can you double check?

Looks like I misspoke.  7.3.2 does not link on 10.3 either.  I don't
know if this makes things better or worse.  Here's the output from
7.3.2.  7.3.4 says the same thing.

gcc -no-cpp-precomp -O3 -D__APPLE__ -Wall -Wmissing-prototypes
-Wmissing-declarations    access/SUBSYS.o bootstrap/SUBSYS.o
catalog/SUBSYS.o parser/SUBSYS.o commands/SUBSYS.o executor/SUBSYS.o
lib/SUBSYS.o libpq/SUBSYS.o main/SUBSYS.o nodes/SUBSYS.o
optimizer/SUBSYS.o port/SUBSYS.o postmaster/SUBSYS.o regex/SUBSYS.o
rewrite/SUBSYS.o storage/SUBSYS.o tcop/SUBSYS.o utils/SUBSYS.o -lz
-lreadline -ltermcap -lresolv -ldl -lm  -o postgres
ld: Undefined symbols:
_tas
make[2]: *** [postgres] Error 1

I purposely defined -O3 and -D__APPLE__ when I ./configure-d.  With or
w/o -D__APPLE__ things still fail.  I had a whole mess of things going
wrong with my 10.3 beta box (including a hardware problem), and I must
have gotten things confused.  Again, 7.3.2 doesn't link either.

I don't think the OS X 10.3 betas are readily available (I've payed to
be in Apple's developer program), so if you don't have access to 10.3
but have some idea as to what would cause this problem with tas, I'll
do whatever I can to help test.

eric

On Sep 21, 2003, at 9:03 PM, Tom Lane wrote:
> That makes no sense at all --- AFAICT there were *no* darwin or ppc
> specific changes between 7.3.2 and 7.3.4.  Can you double check?

Not really knowing what I'm doing, I took s_lock.c and s_lock.h from
7.4beta3, copied 'em into the 7.3.4 src tree, and recompiled.
Surprisingly, everything compiled and 7.3.4 seems to be running okay (I
haven't run the regression tests or tried to stress test it yet).  I'm
not sure this contributes to this thread in a meaningful way, but I
thought it was worth mentioning.

eric

Marko Karppinen <karppinen@pobox.com> writes:
> While we're on a Mac-related note, I managed to compile PostgreSQL on  
> Mac OS X 10.2 Jaguar with two-level namespace support.  
> <http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/ 
> TwoLevelNamespaces.html> This is quite useful, so I hope you can look  
> into incorporating the changes.

> http://www.markokarppinen.com/pg-two-level-namespace.diff

Three immediate questions:

1.  Doesn't this break backward compatibility with Darwin 1.0-1.2?   (If so, does anyone care anymore?)

2.  Why is the postgres executable added to LINK.shared?  Doesn't this   break building shlibs that are not intended to
belinked into the   backend (eg libpq)?
 

3.  What's with the WITHOUT_DARWIN_BUNDLE_LOADER hack for ecpg?
        regards, tom lane

On 23.9.2003, at 19:01, Tom Lane wrote:
> 1.  Doesn't this break backward compatibility with Darwin 1.0-1.2?
>     (If so, does anyone care anymore?)

No. The two-level namespace feature was added in 10.1, and the
-flat_namespace hack was included in pgsql to unbreak the compile
on 10.1 and later. This change hence doesn't have any effect on
10.0 / Darwin 1.x -- except for gcc warnings about unknown flags.

The other change we discussed, ie. using -no-cpp-precomp instead
of -traditional-cpp, will break Darwin 1.x support.

> 2.  Why is the postgres executable added to LINK.shared?  Doesn't this
>     break building shlibs that are not intended to be linked into the
>     backend (eg libpq)?

When linking a shared object, the Darwin linker wants to know that all
the symbols are accounted for -- it doesn't allow undefined symbols.
The bundle_loader flag gives the linker a hint: these symbols will be
provided by the application that loads us, so you don't need raise
an error about them being undefined.

In other words, the -bundle_loader /path/to/postgresql doesn't affect
the build results at all. In the case where there are no undefined 
symbols
(like in libpq), the flag is ignored.

> 3.  What's with the WITHOUT_DARWIN_BUNDLE_LOADER hack for ecpg?

The linker doesn't like symbols that are defined both in the shared 
object
that's being linked and the bundle_loader binary. ecpg contains symbols
that are in the postgresql binary, too, leading to the link failing with
multiply-defined errors.

The easiest way for me to fix this -- I don't really know my way around 
the
source base -- was to specify by hand that the -bundle-loader flag isn't
used on these occasions. Perhaps a more prudent way would be to
examine which shared objects actually have undefined symbols and
use the bundle-loader flag only when linking them, but I think it'd lead
to much more maintenance and bloated Makefiles.

cheers
mk

> Marko Karppinen <karppinen@pobox.com> writes:
>> While we're on a Mac-related note, I managed to compile PostgreSQL on
>> Mac OS X 10.2 Jaguar with two-level namespace support.
>> <http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/
>> TwoLevelNamespaces.html> This is quite useful, so I hope you can look
>> into incorporating the changes.
>
>> http://www.markokarppinen.com/pg-two-level-namespace.diff

Marko Karppinen <karppinen@pobox.com> writes:
>> 3.  What's with the WITHOUT_DARWIN_BUNDLE_LOADER hack for ecpg?

> The linker doesn't like symbols that are defined both in the shared 
> object
> that's being linked and the bundle_loader binary. ecpg contains symbols
> that are in the postgresql binary, too, leading to the link failing with
> multiply-defined errors.

> The easiest way for me to fix this -- I don't really know my way around 
> the
> source base -- was to specify by hand that the -bundle-loader flag isn't
> used on these occasions.

I think the right fix is that "-bundle-loader /path/to/postgres" should
be included in the link only for shlibs that are intended to be loaded
into the backend.  Otherwise you are lying to the linker; the mere fact
that it fails to fail doesn't make it the right thing to do.  (As an
example of how it could go wrong, what if there is a real undefined
symbol problem in libpq, but it happens to match some backend symbol?)

We used to have a similar requirement for some other platforms (AIX
I think), and there was a macro named something like BE_SHLIBS to
include the right platform-dependent stuff for such shlibs.  Not sure
what happened to it.  Peter, do you remember that?
        regards, tom lane

Tom Lane writes:

> We used to have a similar requirement for some other platforms (AIX
> I think), and there was a macro named something like BE_SHLIBS to
> include the right platform-dependent stuff for such shlibs.  Not sure
> what happened to it.  Peter, do you remember that?

BE_DLLLIBS; see Makefile.cygwin for example.  (AIX has a similar
requirement, but handles it differently for bizarre reasons.)

Personally, I think the two-level namespace feature is the opposite of
useful and we should stick with -flat_namespace, but I might have to give
in in the interest of having PostgreSQL behave like other packages on that
system.

-- 
Peter Eisentraut   peter_e@gmx.net

Peter Eisentraut <peter_e@gmx.net> writes:
>> Peter, do you remember that?

> BE_DLLLIBS; see Makefile.cygwin for example.  (AIX has a similar
> requirement, but handles it differently for bizarre reasons.)

Right, thanks.

> Personally, I think the two-level namespace feature is the opposite of
> useful and we should stick with -flat_namespace, but I might have to give
> in in the interest of having PostgreSQL behave like other packages on that
> system.

What do you have against the two-level namespace stuff?  I find the
arguments at
http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/TwoLevelNamespaces.html
moderately convincing.  It doesn't seem like linking a symbol from
library A instead of the expected library B would ever be a good idea.
And I read the document as warning that Apple isn't promising there will
never be name conflicts across standard libraries.

It is kind of a PITA to have to be careful to link a shlib against
everything it will use at runtime, but since we have some other
supported platforms where that's required anyway, we don't really have
a choice about maintaining the code to do it.  Given that, I'm actually
kind of attracted to converting the Darwin port to become a platform
where that's required, because Darwin is a platform that I have easy
access to and test on fairly regularly (when my laptop is working,
anyway ;-)).  We'd be less likely to suffer bit rot in this respect
if the Darwin port required it too.
        regards, tom lane

Eric Ridge <ebr@tcdi.com> writes:
> I don't think the OS X 10.3 betas are readily available (I've payed to
> be in Apple's developer program), so if you don't have access to 10.3
> but have some idea as to what would cause this problem with tas, I'll
> do whatever I can to help test.

I have verified that CVS tip builds okay on 10.3 beta.  I would
recommend dropping the CVS-tip versions of s_lock.h and s_lock.c
into the 7.3 source tree if you need to get 7.3 working on 10.3.

            regards, tom lane

Marko Karppinen <karppinen@pobox.com> writes:
> While we're on a Mac-related note, I managed to compile PostgreSQL on  
> Mac OS X 10.2 Jaguar with two-level namespace support.  
> <http://developer.apple.com/documentation/ReleaseNotes/DeveloperTools/ 
> TwoLevelNamespaces.html> This is quite useful, so I hope you can look  
> into incorporating the changes.

I have done this (not quite as per your original patch, but it's in).

The page mentioned above suggests using slightly different library
routines for shared-library loading than src/backend/port/dynloader/darwin.c
currently uses.  I am not sure if it's important to change or not.
Comments?

> This involves getting rid of undefined symbols in all shared libraries  
> (or "bundles") that get built. I tested this with "./configure" and  
> "./configure --with-tcl" but didn't try the jdbc or python extensions.  

I can report that plpython compiles.  Don't seem to have a working ant
on my 10.2 installation, though, so I can't check jdbc.
        regards, tom lane

"Eric B. Ridge" <ebr@tcdi.com> writes:
> Were you ever able to figure out why 7.3.4 wouldn't build?

Didn't really look, since the 7.4 inlined version of TAS is a better
solution anyway.

            regards, tom lane

Eric B.Ridge wrote:

> On Sep 27, 2003, at 3:43 PM, Tom Lane wrote:
>
>> Eric Ridge <ebr@tcdi.com> writes:
>>
>>> I don't think the OS X 10.3 betas are readily available (I've payed to
>>> be in Apple's developer program), so if you don't have access to 10.3
>>> but have some idea as to what would cause this problem with tas, I'll
>>> do whatever I can to help test.
>>
>>
>> I have verified that CVS tip builds okay on 10.3 beta.  I would
>> recommend dropping the CVS-tip versions of s_lock.h and s_lock.c
>> into the 7.3 source tree if you need to get 7.3 working on 10.3.
>
>
> Using s_lock.c and .h from at least 7.4 works too.
>
> Were you ever able to figure out why 7.3.4 wouldn't build?
>
> eric
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings

if you execute 'select_gcc 3.1' as root it should change your default
GCC to a compiler that works.

On Sep 27, 2003, at 3:43 PM, Tom Lane wrote:

> Eric Ridge <ebr@tcdi.com> writes:
>> I don't think the OS X 10.3 betas are readily available (I've payed to
>> be in Apple's developer program), so if you don't have access to 10.3
>> but have some idea as to what would cause this problem with tas, I'll
>> do whatever I can to help test.
>
> I have verified that CVS tip builds okay on 10.3 beta.  I would
> recommend dropping the CVS-tip versions of s_lock.h and s_lock.c
> into the 7.3 source tree if you need to get 7.3 working on 10.3.

Using s_lock.c and .h from at least 7.4 works too.

Were you ever able to figure out why 7.3.4 wouldn't build?

eric

> if you execute 'select_gcc 3.1' as root it should change your default
> GCC to a compiler that works.

That fixes the problems with -traditional-cpp v/s -no-cpp-precomp, but
it doesn't fix:
    ld: Undefined symbol
    _tas

eric