Thread: install-strip causes dyld errors on OS X

install-strip causes dyld errors on OS X

From
"Meredith L. Patterson"
Date:
Hi all,

Three years ago, Andrew MacRae logged the following bug:

http://archives.postgresql.org/pgsql-bugs/2004-02/msg00042.php

In brief, when installing on OS X with "make install-strip", 
installation goes fine, but initdb dies here:

creating conversions... ERROR:  could not load library
"/usr/local/pgsql/lib/ascii_and_mic.so": dyld: 
/usr/local/pgsql/bin/postgres
Undefined symbols:        /usr/local/pgsql/lib/ascii_and_mic.so undefined reference to
_pg_ascii2mic expected to be defined in the executable

I encountered this problem today with OS X 10.4 and CVS HEAD, and came 
up with a few possible solutions. The problem has to do with the 
difference between shared libraries and dynamic libraries, and how strip 
behaves with them on OS X.

Here's how it works: When called on an executable which uses dyld, strip 
silently passes itself the -u (save all undefined symbols) and -r (save 
all dynamically-referenced symbols) options. However, the 
internationalization libraries (and a few others, like libplperl and 
libplpython) are compiled as shared libraries (.so as opposed to 
.dylib), so strip doesn't know that those libraries contain undefined 
symbols and are expecting to acquire them from the executable.

I see three possible fixes:

1) Patch config/install-sh such that on OS X, install-strip calls 'strip 
-x'. This removes local symbols only.
2) Determine which symbols are global and undefined in the shared 
libraries that are built, make a list of them, and call 'strip -u -r -s 
symbols_to_keep'. This saves undefined symbols, dynamically referenced 
symbols, and symbols that the shared libraries need.
3) Change the OS X build such that the i18n libs and other shared libs 
are instead built as dylibs.

I've tested (1) as far as getting through initdb, starting the 
postmaster and a few quick things with psql. It would be about a 
four-line patch to install-sh (check for the OS, set stripcmd 
appropriately). It might not leave the resulting binary *as* small as it 
could be, but it certainly works.

(2) sounds like a giant pain in the ass. It's scriptable and could be 
added to the build, but the build takes pretty long as it is and I can't 
imagine anyone wants to make it longer. Unless there's a good reason, 
I'm pre-emptively -1'ing this one.

I haven't done any looking into (3) yet. Why *are* the i18n libs built 
as shared on OS X, anyway?

Cheers,
--mlp
_____________________
Meredith L. Patterson
Founder and CTO
Osogato, Inc.


Re: install-strip causes dyld errors on OS X

From
Tom Lane
Date:
"Meredith L. Patterson" <mlp@osogato.com> writes:
> In brief, when installing on OS X with "make install-strip", 
> installation goes fine, but initdb dies here:
> ...
> I see three possible fixes:
> 1) Patch config/install-sh such that on OS X, install-strip calls 'strip 
> -x'. This removes local symbols only.

+1 on this one --- seems about the right level of effort, considering
that install-strip isn't exactly an encouraged option anyway.  (Does
anyone still care about the amount of disk space involved?  And it makes
debugging problems infinitely harder.)

Just for the record, I'd like to put forward the alternative solution
of removing support for install-strip.  But assuming there are
objections to that, please send in the mentioned 4-line patch ...
        regards, tom lane


Re: install-strip causes dyld errors on OS X

From
Alvaro Herrera
Date:
Tom Lane wrote:

> Just for the record, I'd like to put forward the alternative solution
> of removing support for install-strip.

Seconded.

It would also be good to make --enable-debug the default.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: install-strip causes dyld errors on OS X

From
Greg Smith
Date:
On Tue, 30 Oct 2007, Alvaro Herrera wrote:

> It would also be good to make --enable-debug the default.

The last time I did a build with both --enable-debug and --enable-cassert, 
I was seeing pgbench results that were significantly (around 40% I think) 
slower than without those two.  I never circled back to assigning blame to 
the individual options, but I'd suggest some caution here before turning 
one of these on by default.  I'd want to see benchmarking on a couple of 
platforms to prove that it doesn't slow things down before making such a 
change.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: install-strip causes dyld errors on OS X

From
"Joshua D. Drake"
Date:
On Tue, 30 Oct 2007 13:00:12 -0400 (EDT)
Greg Smith <gsmith@gregsmith.com> wrote:

> On Tue, 30 Oct 2007, Alvaro Herrera wrote:
>
> > It would also be good to make --enable-debug the default.
>
> The last time I did a build with both --enable-debug and
> --enable-cassert, I was seeing pgbench results that were
> significantly (around 40% I think) slower than without those two.  I
> never circled back to assigning blame to the individual options, but
> I'd suggest some caution here before turning one of these on by
> default.

Assert certainly has a pretty big penalty but debug isn't nearly as bad.

Joshua D. Drake
 I'd want to see benchmarking on a couple of platforms to
> prove that it doesn't slow things down before making such a change.
>
> --
> * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore,
> MD
>
> ---------------------------(end of
> broadcast)--------------------------- TIP 4: Have you searched our
> list archives?
>
>                http://archives.postgresql.org
>


--
     === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/        UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


Re: install-strip causes dyld errors on OS X

From
Heikki Linnakangas
Date:
Greg Smith wrote:
> On Tue, 30 Oct 2007, Alvaro Herrera wrote:
> 
>> It would also be good to make --enable-debug the default.
> 
> The last time I did a build with both --enable-debug and 
> --enable-cassert, I was seeing pgbench results that were significantly 
> (around 40% I think) slower than without those two.  I never circled 
> back to assigning blame to the individual options, but I'd suggest some 
> caution here before turning one of these on by default.  I'd want to see 
> benchmarking on a couple of platforms to prove that it doesn't slow 
> things down before making such a change.

--enable-cassert is the culprit. You don't want to enable that if you're 
doing performance testing. I haven't seen a slowdown from 
--enable-debug, though I haven't specifically tested that.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


Re: install-strip causes dyld errors on OS X

From
Tom Lane
Date:
Greg Smith <gsmith@gregsmith.com> writes:
> On Tue, 30 Oct 2007, Alvaro Herrera wrote:
>> It would also be good to make --enable-debug the default.

> The last time I did a build with both --enable-debug and --enable-cassert, 
> I was seeing pgbench results that were significantly (around 40% I think) 
> slower than without those two.

Yeah, cassert is expensive (mostly it's the CLOBBER_FREED_MEMORY part,
which you can disable separately if you have a mind to).  --enable-debug
is supposed to be free when using gcc, though I've never tried to verify
that.  We can *not* make --enable-debug the default on non-gcc
compilers, though, and that makes me uncomfortable about turning it on
by default for gcc ... platform-specific defaults don't sound good.
        regards, tom lane