Thread: Cleaning up multiply-defined-symbol warnings on OS X

Recent versions of Darwin spew a lot of multiply-defined-symbol warnings
while building Postgres, mostly because the programs in src/bin import
both libpq and libpgport, and libpq includes copies of many of the
libpgport files.  Since the libpgport routines aren't officially part of
the libpq API, it'd be better if those symbols weren't visible from
outside libpq.  The attached patch makes this so, using the already
existing exports.txt list as the definition of libpq's official API.

I found while testing the patch that we have one API leak in current
sources: ecpglib is depending on the libpq-exported pqGetpwuid()
because src/port/path.c requires it but src/port/thread.c isn't included
into ecpglib.  The patch corrects this oversight.  This BTW is
sufficient reason for a libpq major version bump if we apply this patch;
we learned that lesson last time we fixed an API leak.  (Did we already
bump libpq's major version for 8.2?  I don't recall.)

The reason I didn't go ahead and apply this is that I'm wondering if
there's some more-portable solution that would work for other platforms
besides Darwin.  I seem to recall that we've discussed the point before,
but no patch has been forthcoming.

Comments?

            regards, tom lane

Index: src/Makefile.shlib
===================================================================
RCS file: /cvsroot/pgsql/src/Makefile.shlib,v
retrieving revision 1.103
diff -c -r1.103 Makefile.shlib
*** src/Makefile.shlib    19 Apr 2006 16:32:08 -0000    1.103
--- src/Makefile.shlib    20 Apr 2006 00:56:38 -0000
***************
*** 107,117 ****
    ifeq ($(DLTYPE), library)
      # linkable library
      DLSUFFIX        := .dylib
!     LINK.shared        = $(COMPILER) -dynamiclib -install_name $(libdir)/lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
$(version_link)-multiply_defined suppress 
    else
      # loadable module (default case)
      DLSUFFIX        := .so
!     LINK.shared        = $(COMPILER) -bundle
    endif
    shlib            = lib$(NAME).$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION)$(DLSUFFIX)
    shlib_major        = lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
--- 107,117 ----
    ifeq ($(DLTYPE), library)
      # linkable library
      DLSUFFIX        := .dylib
!     LINK.shared        = $(COMPILER) -dynamiclib -install_name $(libdir)/lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
$(version_link)$(exported_symbols_list) -multiply_defined suppress 
    else
      # loadable module (default case)
      DLSUFFIX        := .so
!     LINK.shared        = $(COMPILER) -bundle -multiply_defined suppress
    endif
    shlib            = lib$(NAME).$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION)$(DLSUFFIX)
    shlib_major        = lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
Index: src/interfaces/libpq/Makefile
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/libpq/Makefile,v
retrieving revision 1.143
diff -c -r1.143 Makefile
*** src/interfaces/libpq/Makefile    11 Apr 2006 20:26:40 -0000    1.143
--- src/interfaces/libpq/Makefile    20 Apr 2006 00:56:39 -0000
***************
*** 125,130 ****
--- 125,143 ----
      echo '; Aliases for MS compatible names' >> $@
      sed -e '/^#/d' -e 's/^\(.* \)\([0-9][0-9]*\)/    \1= _\1/' < $< | sed 's/ *$$//' >> $@

+ # On Darwin, restrict the symbols exported by the library to just the
+ # official list, so as to avoid multiply-defined-symbol warnings in
+ # programs that use libpgport along with libpq.
+
+ ifeq ($(PORTNAME), darwin)
+ $(shlib): exports.darwin
+
+ exports.darwin: exports.txt
+     $(AWK) '/^[^#]/ {printf "_%s\n",$$1}' $< >$@
+
+ exported_symbols_list = -exported_symbols_list exports.darwin
+ endif
+
  # depend on Makefile.global to force rebuild on re-run of configure
  $(srcdir)/libpq.rc: libpq.rc.in $(top_builddir)/src/Makefile.global
      sed -e 's/\(VERSION.*\),0 *$$/\1,'`date '+%y%j' | sed 's/^0*//'`'/' < $< > $@
***************
*** 147,153 ****
      rm -f '$(DESTDIR)$(includedir)/libpq-fe.h' '$(DESTDIR)$(includedir_internal)/libpq-int.h'
'$(DESTDIR)$(includedir_internal)/pqexpbuffer.h''$(DESTDIR)$(datadir)/pg_service.conf.sample' 

  clean distclean: clean-lib
!     rm -f $(OBJS) pg_config_paths.h crypt.c getaddrinfo.c inet_aton.c noblock.c pgstrcasecmp.c snprintf.c strerror.c
open.cthread.c md5.c ip.c encnames.c wchar.c pthread.h 
      rm -f pg_config_paths.h    # Might be left over from a Win32 client-only build

  maintainer-clean: distclean
--- 160,166 ----
      rm -f '$(DESTDIR)$(includedir)/libpq-fe.h' '$(DESTDIR)$(includedir_internal)/libpq-int.h'
'$(DESTDIR)$(includedir_internal)/pqexpbuffer.h''$(DESTDIR)$(datadir)/pg_service.conf.sample' 

  clean distclean: clean-lib
!     rm -f $(OBJS) pg_config_paths.h crypt.c getaddrinfo.c inet_aton.c noblock.c pgstrcasecmp.c snprintf.c strerror.c
open.cthread.c md5.c ip.c encnames.c wchar.c pthread.h exports.darwin 
      rm -f pg_config_paths.h    # Might be left over from a Win32 client-only build

  maintainer-clean: distclean
Index: src/interfaces/ecpg/ecpglib/Makefile
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/ecpg/ecpglib/Makefile,v
retrieving revision 1.38
diff -c -r1.38 Makefile
*** src/interfaces/ecpg/ecpglib/Makefile    17 Jan 2006 19:49:23 -0000    1.38
--- src/interfaces/ecpg/ecpglib/Makefile    20 Apr 2006 00:56:39 -0000
***************
*** 25,31 ****
  LIBS := $(filter-out -lpgport, $(LIBS))

  OBJS= execute.o typename.o descriptor.o data.o error.o prepare.o memory.o \
!     connect.o misc.o path.o exec.o \
      $(filter snprintf.o, $(LIBOBJS))

  SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
--- 25,31 ----
  LIBS := $(filter-out -lpgport, $(LIBS))

  OBJS= execute.o typename.o descriptor.o data.o error.o prepare.o memory.o \
!     connect.o misc.o path.o exec.o thread.o \
      $(filter snprintf.o, $(LIBOBJS))

  SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
***************
*** 46,52 ****
  # necessarily use the same object files as the backend uses. Instead,
  # symlink the source files in here and build our own object file.

! path.c exec.c snprintf.c: % : $(top_srcdir)/src/port/%
      rm -f $@ && $(LN_S) $< .

  path.o: path.c $(top_builddir)/src/port/pg_config_paths.h
--- 46,52 ----
  # necessarily use the same object files as the backend uses. Instead,
  # symlink the source files in here and build our own object file.

! path.c exec.c snprintf.c thread.c: % : $(top_srcdir)/src/port/%
      rm -f $@ && $(LN_S) $< .

  path.o: path.c $(top_builddir)/src/port/pg_config_paths.h
***************
*** 62,68 ****
  uninstall: uninstall-lib

  clean distclean maintainer-clean: clean-lib
!     rm -f $(OBJS) path.c exec.c snprintf.c

  depend dep:
      $(CC) -MM $(CFLAGS) *.c >depend
--- 62,68 ----
  uninstall: uninstall-lib

  clean distclean maintainer-clean: clean-lib
!     rm -f $(OBJS) path.c exec.c snprintf.c thread.c

  depend dep:
      $(CC) -MM $(CFLAGS) *.c >depend

We have two approaches for dealing with pgport leakage.  For libraries,
we remove libpgport from the link line and recompile our own object
files:

    # Need to recompile any libpgport object files
    LIBS := $(filter-out -lpgport, $(LIBS))

    OBJS= execute.o typename.o descriptor.o data.o error.o prepare.o memory.o \
            connect.o misc.o path.o exec.o \
            $(filter snprintf.o, $(LIBOBJS))

The problem is that there is no testing of which object files need to be
added, but fortunately Win32 has the dllexport which makes libpq export
_only_ functions we define, so if we forget something, it shows up as a
Win32 link error.

For applications, we have them pull symbols from libpgport first, and
are not bound to specific libpgport functions being in libpq.
Makefile.global has:

    # Force clients to pull symbols from the non-shared library libpgport
    # rather than pulling some libpgport symbols from libpq just because
    # libpq uses those functions too.  This makes applications less
    # dependent on changes in libpq's usage of pgport.  To do this we link to
    # pgport before libpq.  This does cause duplicate -lpgport's to appear
    # on client link lines.
    ifdef PGXS
    libpq_pgport = -L$(libdir) -lpgport $(libpq)
    else
    libpq_pgport = -L$(top_builddir)/src/port -lpgport $(libpq)
    endif

That is our _portable_ way to do it.

So, technically we have everything covered, at least since 8.0.2.

The problem is that the call to thread.c::pqGetpwuid() happens in a
#ifndef WIN32 block, so the function is not seen by Win32, so we don't
get a compile error.

I am thinking your patch is a good idea because it will give us a
non-Win32 platform that has the _check_ behavior.  We do have to bump
the major for ecpg libs for this, but not libpq.

---------------------------------------------------------------------------

Tom Lane wrote:
> Recent versions of Darwin spew a lot of multiply-defined-symbol warnings
> while building Postgres, mostly because the programs in src/bin import
> both libpq and libpgport, and libpq includes copies of many of the
> libpgport files.  Since the libpgport routines aren't officially part of
> the libpq API, it'd be better if those symbols weren't visible from
> outside libpq.  The attached patch makes this so, using the already
> existing exports.txt list as the definition of libpq's official API.
>
> I found while testing the patch that we have one API leak in current
> sources: ecpglib is depending on the libpq-exported pqGetpwuid()
> because src/port/path.c requires it but src/port/thread.c isn't included
> into ecpglib.  The patch corrects this oversight.  This BTW is
> sufficient reason for a libpq major version bump if we apply this patch;
> we learned that lesson last time we fixed an API leak.  (Did we already
> bump libpq's major version for 8.2?  I don't recall.)
>
> The reason I didn't go ahead and apply this is that I'm wondering if
> there's some more-portable solution that would work for other platforms
> besides Darwin.  I seem to recall that we've discussed the point before,
> but no patch has been forthcoming.
>
> Comments?
>
>             regards, tom lane
>

Content-Description: darwin-symbols.patch

> Index: src/Makefile.shlib
> ===================================================================
> RCS file: /cvsroot/pgsql/src/Makefile.shlib,v
> retrieving revision 1.103
> diff -c -r1.103 Makefile.shlib
> *** src/Makefile.shlib    19 Apr 2006 16:32:08 -0000    1.103
> --- src/Makefile.shlib    20 Apr 2006 00:56:38 -0000
> ***************
> *** 107,117 ****
>     ifeq ($(DLTYPE), library)
>       # linkable library
>       DLSUFFIX        := .dylib
> !     LINK.shared        = $(COMPILER) -dynamiclib -install_name $(libdir)/lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
$(version_link)-multiply_defined suppress 
>     else
>       # loadable module (default case)
>       DLSUFFIX        := .so
> !     LINK.shared        = $(COMPILER) -bundle
>     endif
>     shlib            = lib$(NAME).$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION)$(DLSUFFIX)
>     shlib_major        = lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
> --- 107,117 ----
>     ifeq ($(DLTYPE), library)
>       # linkable library
>       DLSUFFIX        := .dylib
> !     LINK.shared        = $(COMPILER) -dynamiclib -install_name $(libdir)/lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
$(version_link)$(exported_symbols_list) -multiply_defined suppress 
>     else
>       # loadable module (default case)
>       DLSUFFIX        := .so
> !     LINK.shared        = $(COMPILER) -bundle -multiply_defined suppress
>     endif
>     shlib            = lib$(NAME).$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION)$(DLSUFFIX)
>     shlib_major        = lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)
> Index: src/interfaces/libpq/Makefile
> ===================================================================
> RCS file: /cvsroot/pgsql/src/interfaces/libpq/Makefile,v
> retrieving revision 1.143
> diff -c -r1.143 Makefile
> *** src/interfaces/libpq/Makefile    11 Apr 2006 20:26:40 -0000    1.143
> --- src/interfaces/libpq/Makefile    20 Apr 2006 00:56:39 -0000
> ***************
> *** 125,130 ****
> --- 125,143 ----
>       echo '; Aliases for MS compatible names' >> $@
>       sed -e '/^#/d' -e 's/^\(.* \)\([0-9][0-9]*\)/    \1= _\1/' < $< | sed 's/ *$$//' >> $@
>
> + # On Darwin, restrict the symbols exported by the library to just the
> + # official list, so as to avoid multiply-defined-symbol warnings in
> + # programs that use libpgport along with libpq.
> +
> + ifeq ($(PORTNAME), darwin)
> + $(shlib): exports.darwin
> +
> + exports.darwin: exports.txt
> +     $(AWK) '/^[^#]/ {printf "_%s\n",$$1}' $< >$@
> +
> + exported_symbols_list = -exported_symbols_list exports.darwin
> + endif
> +
>   # depend on Makefile.global to force rebuild on re-run of configure
>   $(srcdir)/libpq.rc: libpq.rc.in $(top_builddir)/src/Makefile.global
>       sed -e 's/\(VERSION.*\),0 *$$/\1,'`date '+%y%j' | sed 's/^0*//'`'/' < $< > $@
> ***************
> *** 147,153 ****
>       rm -f '$(DESTDIR)$(includedir)/libpq-fe.h' '$(DESTDIR)$(includedir_internal)/libpq-int.h'
'$(DESTDIR)$(includedir_internal)/pqexpbuffer.h''$(DESTDIR)$(datadir)/pg_service.conf.sample' 
>
>   clean distclean: clean-lib
> !     rm -f $(OBJS) pg_config_paths.h crypt.c getaddrinfo.c inet_aton.c noblock.c pgstrcasecmp.c snprintf.c
strerror.copen.c thread.c md5.c ip.c encnames.c wchar.c pthread.h 
>       rm -f pg_config_paths.h    # Might be left over from a Win32 client-only build
>
>   maintainer-clean: distclean
> --- 160,166 ----
>       rm -f '$(DESTDIR)$(includedir)/libpq-fe.h' '$(DESTDIR)$(includedir_internal)/libpq-int.h'
'$(DESTDIR)$(includedir_internal)/pqexpbuffer.h''$(DESTDIR)$(datadir)/pg_service.conf.sample' 
>
>   clean distclean: clean-lib
> !     rm -f $(OBJS) pg_config_paths.h crypt.c getaddrinfo.c inet_aton.c noblock.c pgstrcasecmp.c snprintf.c
strerror.copen.c thread.c md5.c ip.c encnames.c wchar.c pthread.h exports.darwin 
>       rm -f pg_config_paths.h    # Might be left over from a Win32 client-only build
>
>   maintainer-clean: distclean
> Index: src/interfaces/ecpg/ecpglib/Makefile
> ===================================================================
> RCS file: /cvsroot/pgsql/src/interfaces/ecpg/ecpglib/Makefile,v
> retrieving revision 1.38
> diff -c -r1.38 Makefile
> *** src/interfaces/ecpg/ecpglib/Makefile    17 Jan 2006 19:49:23 -0000    1.38
> --- src/interfaces/ecpg/ecpglib/Makefile    20 Apr 2006 00:56:39 -0000
> ***************
> *** 25,31 ****
>   LIBS := $(filter-out -lpgport, $(LIBS))
>
>   OBJS= execute.o typename.o descriptor.o data.o error.o prepare.o memory.o \
> !     connect.o misc.o path.o exec.o \
>       $(filter snprintf.o, $(LIBOBJS))
>
>   SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
> --- 25,31 ----
>   LIBS := $(filter-out -lpgport, $(LIBS))
>
>   OBJS= execute.o typename.o descriptor.o data.o error.o prepare.o memory.o \
> !     connect.o misc.o path.o exec.o thread.o \
>       $(filter snprintf.o, $(LIBOBJS))
>
>   SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
> ***************
> *** 46,52 ****
>   # necessarily use the same object files as the backend uses. Instead,
>   # symlink the source files in here and build our own object file.
>
> ! path.c exec.c snprintf.c: % : $(top_srcdir)/src/port/%
>       rm -f $@ && $(LN_S) $< .
>
>   path.o: path.c $(top_builddir)/src/port/pg_config_paths.h
> --- 46,52 ----
>   # necessarily use the same object files as the backend uses. Instead,
>   # symlink the source files in here and build our own object file.
>
> ! path.c exec.c snprintf.c thread.c: % : $(top_srcdir)/src/port/%
>       rm -f $@ && $(LN_S) $< .
>
>   path.o: path.c $(top_builddir)/src/port/pg_config_paths.h
> ***************
> *** 62,68 ****
>   uninstall: uninstall-lib
>
>   clean distclean maintainer-clean: clean-lib
> !     rm -f $(OBJS) path.c exec.c snprintf.c
>
>   depend dep:
>       $(CC) -MM $(CFLAGS) *.c >depend
> --- 62,68 ----
>   uninstall: uninstall-lib
>
>   clean distclean maintainer-clean: clean-lib
> !     rm -f $(OBJS) path.c exec.c snprintf.c thread.c
>
>   depend dep:
>       $(CC) -MM $(CFLAGS) *.c >depend

>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>        subscribe-nomail command to majordomo@postgresql.org so that your
>        message can get through to the mailing list cleanly

--
  Bruce Momjian   http://candle.pha.pa.us
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I am thinking your patch is a good idea because it will give us a
> non-Win32 platform that has the _check_ behavior.  We do have to bump
> the major for ecpg libs for this, but not libpq.

No, because if 8.2 ships with a libpq.so.4 that doesn't expose
pqGetpwuid, it will look like pre-8.2 versions of libecpg will work
with that libpq.  Which they won't.  You can't remove exported
functions (whether the export was intentional or no) without a
major version bump.

            regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I am thinking your patch is a good idea because it will give us a
> > non-Win32 platform that has the _check_ behavior.  We do have to bump
> > the major for ecpg libs for this, but not libpq.
>
> No, because if 8.2 ships with a libpq.so.4 that doesn't expose
> pqGetpwuid, it will look like pre-8.2 versions of libecpg will work
> with that libpq.  Which they won't.  You can't remove exported
> functions (whether the export was intentional or no) without a
> major version bump.

OK, right.

--
  Bruce Momjian   http://candle.pha.pa.us
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> The problem is that the call to thread.c::pqGetpwuid() happens in a
> #ifndef WIN32 block, so the function is not seen by Win32, so we don't
> get a compile error.

> I am thinking your patch is a good idea because it will give us a
> non-Win32 platform that has the _check_ behavior.  We do have to bump
> the major for ecpg libs for this, but not libpq.

I just came across some documentation claiming that it's possible to do
similar exported-symbol filtering in Linux:
    http://people.redhat.com/%7edrepper/dsohowto.pdf
I'm going to see if I can make that work along the same lines as my
Darwin patch.  If we can get it going on both Linux and Darwin then
I think we'll have critical mass to make the change.

            regards, tom lane

Tom Lane wrote:
> I just came across some documentation claiming that it's possible to do
> similar exported-symbol filtering in Linux:
>         http://people.redhat.com/%7edrepper/dsohowto.pdf
> I'm going to see if I can make that work along the same lines as my
> Darwin patch.  If we can get it going on both Linux and Darwin then
> I think we'll have critical mass to make the change.

Now that 95% of this patch [1] (currently in the hold queue) has been
implemented, perhaps we should go the final 5% and version the symbols
to fix the ambiguous symbol problem forever (on linux at least, though
anywhere with the GCC toolchain will work).

Have a nice day,

[1] http://archives.postgresql.org/pgsql-patches/2005-10/msg00166.php
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Martijn van Oosterhout <kleptog@svana.org> writes:
> Now that 95% of this patch [1] (currently in the hold queue) has been
> implemented, perhaps we should go the final 5% and version the symbols
> to fix the ambiguous symbol problem forever (on linux at least, though
> anywhere with the GCC toolchain will work).

I was thinking about that, but came away unconvinced it's worth doing
considering we are not a Linux-only system.  Is there any advantage
in having symbol versioning on Linux when we'll still have the same
API-change constraints everywhere else?

I had actually forgotten about your earlier patch :-(

            regards, tom lane

On Sat, Apr 29, 2006 at 03:07:30PM -0400, Tom Lane wrote:
> Martijn van Oosterhout <kleptog@svana.org> writes:
> > Now that 95% of this patch [1] (currently in the hold queue) has been
> > implemented, perhaps we should go the final 5% and version the symbols
> > to fix the ambiguous symbol problem forever (on linux at least, though
> > anywhere with the GCC toolchain will work).
>
> I was thinking about that, but came away unconvinced it's worth doing
> considering we are not a Linux-only system.  Is there any advantage
> in having symbol versioning on Linux when we'll still have the same
> API-change constraints everywhere else?

What it does is remove the restriction that any one program can only
use (directly or indirectly) one version of libpq at any moment.
Programs can use indirectly postgres via PAM or NSS or other such
pluggable interfaces. Currently PAM and NSS must be using the same
version of libpq as any program that might call them or everything
blows up.

Versioning is basically (amongst other things) a way of ensuring that a
program runs against the same version of the library it was compiled
against. Just renaming a library won't confuse it for example.

It's more a preventative measure than anything else. As postgres gets
integrated into more things the chance of symbols getting mislinked
increases.

> I had actually forgotten about your earlier patch :-(

I figured as much :)

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Am Samstag, 29. April 2006 21:27 schrieb Martijn van Oosterhout:
> What it does is remove the restriction that any one program can only
> use (directly or indirectly) one version of libpq at any moment.
> Programs can use indirectly postgres via PAM or NSS or other such
> pluggable interfaces. Currently PAM and NSS must be using the same
> version of libpq as any program that might call them or everything
> blows up.

Symbol versioning only helps if a function's signature has changed, doesn't
it?  Have we ever done that?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Tue, May 09, 2006 at 10:19:56AM +0200, Peter Eisentraut wrote:
> Am Samstag, 29. April 2006 21:27 schrieb Martijn van Oosterhout:
> > What it does is remove the restriction that any one program can only
> > use (directly or indirectly) one version of libpq at any moment.
> > Programs can use indirectly postgres via PAM or NSS or other such
> > pluggable interfaces. Currently PAM and NSS must be using the same
> > version of libpq as any program that might call them or everything
> > blows up.
>
> Symbol versioning only helps if a function's signature has changed, doesn't
> it?  Have we ever done that?

Depends what you mean by signature. The structures of PGconn and
PGresult have changed over time, so if you you pass a PGresult
allocated by libpq4 to a function in libpq3, it'll crash.

Symbol versioning can serve a few purposes:

1. You can have one library supporting several ABIs simultaneously.
glibc does this but that's not what we're interested in.

2. When loading a library, the linker checks both the symbol name and
the symbol version. Thus, it allows two versions of the same library to
co-exist in the same address space simultaneously. This is what we
want.

Basically, if you symbol version, a program compiled against a
particular version with only call functions in that version and won't
be accedently misdirected to another version that happened to be loaded
first...

In the example I posted earlier, say psql starts, compiled against
libpq5. It does a gethostbyname() that is redirected to an NSS module
originally compiled against libpq4. The linker will load libpq4 but
link the actual symbols of the NSS module to libpq5. Any it can't find
there it will find in libpq4. It's this splitting up that will get you.

Loading this way round is unlikely to cause many problems since we
don't tend to remove functions from the API so it may not link any
symbols to libpq4. The more dangerous situation is if the old library
gets loaded first and something uses symbols that didn't exist in the
old version but do in the new version.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Am Dienstag, 9. Mai 2006 10:41 schrieb Martijn van Oosterhout:
> Depends what you mean by signature. The structures of PGconn and
> PGresult have changed over time, so if you you pass a PGresult
> allocated by libpq4 to a function in libpq3, it'll crash.

Symbol versioning only affects functions (and maybe variables?) but not
structs.  Nothing helps in that case.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Tue, May 09, 2006 at 01:50:38PM +0200, Peter Eisentraut wrote:
> Am Dienstag, 9. Mai 2006 10:41 schrieb Martijn van Oosterhout:
> > Depends what you mean by signature. The structures of PGconn and
> > PGresult have changed over time, so if you you pass a PGresult
> > allocated by libpq4 to a function in libpq3, it'll crash.
>
> Symbol versioning only affects functions (and maybe variables?) but not
> structs.  Nothing helps in that case.

Eh? It stops a program expecting libpq4 being linked to libpq3 for any
reason, so the above situation can't happen. You don't need to version
any structs, only the functions using them.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Martijn van Oosterhout <kleptog@svana.org> writes:
> Eh? It stops a program expecting libpq4 being linked to libpq3 for any
> reason, so the above situation can't happen. You don't need to version
> any structs, only the functions using them.

If we have an existing app built against an unversioned libpq, what
happens if we roll in a versioned libpq?  Does it break completely?
That is, is the introduction of versioning by itself an incompatible
ABI break?

            regards, tom lane

On Tue, May 09, 2006 at 09:18:17AM -0400, Tom Lane wrote:
> Martijn van Oosterhout <kleptog@svana.org> writes:
> > Eh? It stops a program expecting libpq4 being linked to libpq3 for any
> > reason, so the above situation can't happen. You don't need to version
> > any structs, only the functions using them.
>
> If we have an existing app built against an unversioned libpq, what
> happens if we roll in a versioned libpq?  Does it break completely?
> That is, is the introduction of versioning by itself an incompatible
> ABI break?

This is a special rules:

- If a program (like psql) requires a symbol but specifies no symbol
version, it will match against that symbol without a version, or if not
found, the symbol with the lowest version number.

So if you slide a versioned library under a program that doesn't know
about it, it will load fine because unversioned symbols match anything.

Here is a quick example of how it works:

$ cat prog.c
extern int func();

int main()
{
  return func();
}
$ cat lib.c
int func()
{
  return 1;
}
$ cat ver_script
VERSION_1 { global: *; };

### Create the various versions ###
$ gcc -shared lib.c -Wl,--version-script=ver_script -Wl,-soname=liblib.so -o liblib_ver.so
$ gcc -shared lib.c -Wl,-soname=liblib.so -o liblib_nover.so
$ gcc -o prog_ver prog.c -L$PWD -llib_ver
$ gcc -o prog_nover prog.c -L$PWD -llib_nover

### Test ###
$ mkdir test
$ cp liblib_nover.so test/liblib.so
$ cp prog_nover test/prog
$ LD_LIBRARY_PATH=$PWD/test test/prog     # Works (both no version)
$ cp liblib_nover.so test/liblib.so
$ cp prog_ver test/prog
$ LD_LIBRARY_PATH=$PWD/test test/prog     # Doesn't work (prog with version, lib without)
test/prog: /tmp/t/test/liblib.so: no version information available (required by test/prog)
$ cp liblib_ver.so test/liblib.so
$ cp prog_ver test/prog
$ LD_LIBRARY_PATH=$PWD/test test/prog     # Works (both with version)
$ cp liblib_ver.so test/liblib.so
$ cp prog_nover test/prog
$ LD_LIBRARY_PATH=$PWD/test test/prog     # Works (prog without version, lib with)

You can make that broken variation work too, if the version is marked
as weak, but I couldn't quickly find out how.

http://people.redhat.com/drepper/symbol-versioning

Hope this helps,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.