Thread: PG Extensions: Must be statically linked?
I'm creating user-defined server extensions, written in C per the manual "31.9. C-Language Functions". Everything workswell, but only if I fully link the .so such that there are *no* unresolved external references at all. Not even thestuff in libstdc++.a can be left out. I've tried setting LD_LIBRARY_PATH everywhere possible, with no luck. Here's themake(1) line I have to use to link: libmyfuncs.so.0.0:gcc -Wall -Wmissing-prototypes -Wpointer-arith \ -Wdeclaration-after-statement -Wold-style-definition-Wendif-labels \ -fno-strict-aliasing -fpic -shared -Wl,-soname,libmyfuncs.so.0 \ $(OBJS) \ $(MYLIB)/lib/libmylibs.a\ /usr/lib/gcc/i386-redhat-linux/3.4.2/libstdc++.a \ -o libmyfuncs.so.0.0 Is this correct? Do Postgres extension need to be fully statically linked? Or is there some configuration that will specifyLD_LIBRARY_PATH (or perhaps a Postgres-specific equivalent). The manual's instructions are good regarding writing code, but don't say much about linking. Thanks, Craig
Craig A. James wrote: > I'm creating user-defined server extensions, written in C per the > manual "31.9. C-Language Functions". Everything works well, but only > if I fully link the .so such that there are *no* unresolved external > references at all. What happens if you don't? -- Peter Eisentraut http://developer.postgresql.org/~petere/
"Craig A. James" <cjames@modgraph-usa.com> writes:
> I'm creating user-defined server extensions, written in C per the
> manual "31.9. C-Language Functions". Everything works well, but only
> if I fully link the .so such that there are *no* unresolved external
> references at all. Not even the stuff in libstdc++.a can be left out.
If you're using libstdc++.a, you are not writing C. There is no support
for C++ in the backend, and I would strongly advise not trying to use
it, as any of C++'s moderately interesting features like exceptions will
not play nicely with the backend environment.
regards, tom lane
Tom Lane wrote: >>I'm creating user-defined server extensions, written in C per the >>manual "31.9. C-Language Functions". Everything works well, but only >>if I fully link the .so such that there are *no* unresolved external >>references at all. Not even the stuff in libstdc++.a can be left out. > > If you're using libstdc++.a, you are not writing C. There is no support > for C++ in the backend, and I would strongly advise not trying to use > it, as any of C++'s moderately interesting features like exceptions will > not play nicely with the backend environment. Unfortunately, we're also using a second library (OpenBabel) that is written in C++. A good portion of the code I've writtenis a wrapper layer that hides the C++ objects and presents a simple C wrapper that works for Postgres. Craig
Craig A. James wrote: > Unfortunately, we're also using a second library (OpenBabel) that is > written in C++. A good portion of the code I've written is a wrapper > layer that hides the C++ objects and presents a simple C wrapper that > works for Postgres. I suggest if you want to get any concrete advice out of this, post us the commands that you execute and the error messages that you get. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Thu, 02 Mar 2006 13:43:35 -0800, Craig A. James wrote: > I'm creating user-defined server extensions, written in C per the manual [snip] > Is this correct? Do Postgres extension need to be fully statically > linked? Or is there some configuration that will specify LD_LIBRARY_PATH > (or perhaps a Postgres-specific equivalent). > Not generally, unless your platform requires it because of restrictive dynamic loader behaviour. contrib/xml2 uses two external libraries (libxml and libxslt) - they are dynamically referenced by the resulting pgxml.so. Have a look at the Makefile for contrib/xml2 for hints on options? > The manual's instructions are good regarding writing code, but don't say > much about linking. > True - but I think the assumption has been that most people wouldn't have any involved linking requirements. Your case doesn't come into this category, but there are useful cases for linking to external function libraries and exposing them to PG. Regards John
Peter Eisentraut wrote: >>Unfortunately, we're also using a second library (OpenBabel) that is >>written in C++. A good portion of the code I've written is a wrapper >>layer that hides the C++ objects and presents a simple C wrapper that >>works for Postgres. > > I suggest if you want to get any concrete advice out of this, post us > the commands that you execute and the error messages that you get. Thanks for your answers -- see below. Based on Peter's and Tom's replies regarding C++, I think you've answered my question: I should be able to do this withoutstatic linking. But the Postgres linker uses the C (not the C++) linker to resolve references, so it's not findingthe C++ libraries. My original question was misleading. I said, "I have to link everything statically...", when in fact what should have saidwas, "If I link statically, it works." I now realize that I'm not linking everything statically, just the OpenBableand C++ libraries, and in fact Postgres is finding the other libraries I need, like libz, libm, and so forth. Here's what happens when I don't statically link the C++ libraries or the OpenBabel libraries: [root]# cp libmyfuncs.so /usr/local/pgsql/lib $ psql -d myfuncs -U postgres Welcome to psql 8.0.3, the PostgreSQL interactive terminal. ... myfuncs=# CREATE FUNCTION myfunc(text, text) RETURNS boolean myfuncs-# AS '/usr/local/pgsql/lib/libmyfuncs.so', 'myfunc' myfuncs-# LANGUAGE 'C' STRICT; ERROR: could not load library "/usr/local/pgsql/lib/libmyfuncs.so": /usr/local/pgsql/lib/libmyfuncs.so: undefined symbol:_ZdlPv That symbol is obviously a C++ mangled name, so it's not finding the C++ library, and the fact that it prints the mangledname suggests that it's a C linker, not a C++ linker. So now my question is: Can I somehow add other directories/libraries to those that Postgres uses? Or is there an optionfor Postgres use the C++ dynamic linker? I don't mind statically linking OpenBabel, but it seems like a bad idea toput the specific version- and system-dependent location of libstdc++.a into my makefiles. (Am I the only guy in the world who has to use a C++ library as part of a Postgres function?) Thanks, Craig
On Fri, Mar 03, 2006 at 07:46:06AM -0800, Craig A. James wrote: > Thanks for your answers -- see below. > > Based on Peter's and Tom's replies regarding C++, I think you've answered > my question: I should be able to do this without static linking. But the > Postgres linker uses the C (not the C++) linker to resolve references, so > it's not finding the C++ libraries. <snip> > So now my question is: Can I somehow add other directories/libraries to > those that Postgres uses? Or is there an option for Postgres use the C++ > dynamic linker? I don't mind statically linking OpenBabel, but it seems > like a bad idea to put the specific version- and system-dependent location > of libstdc++.a into my makefiles. There are a number of ways to indicate which extra libraries to load, but by far the easiest is by specifying them on the link line. Your problem is that when creating a shared library, you are not required to make sure all your external symbols are defined somewhere. You didn't post your compile/link line but if you're using C++ you probably need to use g++ for the linking to include the special C++ libraries. One way to find out what's going on is using readelf: $ readelf -a /bin/bash |grep NEEDED0x00000001 (NEEDED) Shared library: [libncurses.so.5]0x00000001 (NEEDED) Shared library: [libdl.so.2]0x00000001 (NEEDED) Shared library: [libc.so.6] These are the libraries that will be loaded when someone tries to load your shared lib (the above works on any ELF object). The easiest is to specify "--no-undefined" on the link line so the linker checks up front you won't get undefined symbols at run time. Remember it's -Wl,--no-undefined if invoking from gcc/g++. Hope this helps, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
"Craig A. James" <cjames@modgraph-usa.com> writes:
> So now my question is: Can I somehow add other directories/libraries
> to those that Postgres uses?
This is not a Postgres problem, it's a dynamic-linker problem, and
I don't believe there is a different dynamic linker for C++ than C.
Your problem is just to get your shared library marked as needing
libstdc++.so. (ldd on the .so should show what shared libraries it
needs.) On Linux this should all happen pretty much automatically,
at least for libraries that are in the ldconfig search path. I'm
not sure what is going wrong, but you could take a look at
contrib/dblink, which dynamically includes libpq.so (and the backend
certainly does not load libpq.so by default). If that works on your
machine then try to figure out what dblink's Makefile does differently
from yours.
My concern about how nicely libstdc++ will play in the backend
environment still stands though.
regards, tom lane
Tom Lane wrote: > My concern about how nicely libstdc++ will play in the backend > environment still stands though. I have had the same concern, though never any hard evidence of a problem. If the C++ functions are wrapped with "extern C", and all exceptions caught (perhaps converted into error numbers which are then returned from the wrapper functions to the plain-C calling functions), are there any remaining known problems? I have often considered making a C++ allocator which wrapped palloc and pfree, so that I could then use the STL within the backend... Has anyone tried this? mark
Mark Dilger writes: > I have had the same concern, though never any hard evidence of a > problem. If the C++ functions are wrapped with "extern C", and all > exceptions caught (perhaps converted into error numbers which are then > returned from the wrapper functions to the plain-C calling functions), > are there any remaining known problems? I have often considered > making a C++ allocator which wrapped palloc and pfree, so that I could > then use the STL within the backend... > > Has anyone tried this? I did some experiments on a C++ language handler last year (including an allocator and a class loader to spare the extern "C"s/name mangling): <http://archives.postgresql.org/pgsql-general/2005-10/msg01570.php> <news:87acgxjzsl.fsf@gate450.dyndns.org> The remaining issue is the impedance mismatch between longjmp()ing and exceptions. regards, Andreas --