Thread: Is the linking with -lodbc necessary? (--with-odbc)
Hello all! Long story short: Is there a need to link psqlodbcw.so plugin against libodbc.so? Principal problem: That library provides ABI for applications - not for plugins; at least it seem to be like that. I tried to remove this linking by following tweak: | diff --git a/configure.ac b/configure.ac | index 5fb401e..5d75db3 100644 | --- a/configure.ac | +++ b/configure.ac | @@ -65,7 +65,7 @@ fi | # ODBC include and library | # | | -if test "$ODBC_CONFIG" != ""; then | +if false; then | if test "$with_iodbc" != no; then | ODBC_INCLUDE=`${ODBC_CONFIG} --cflags` | CPPFLAGS="$CPPFLAGS ${ODBC_INCLUDE}" And both unixODBC and psqlodbcw.so communicates with each other correctly at the first sight. --------------------------------------------------------------------------- And now the long story. The real problem comes now for 'isql' from unixODBC (and most probably also for others) on s390x architecture in Linux. As you know, unixODBC (== libodbc.so) uses lt_dlopen() (which just wraps dlopen() in described scenario) for opening plugins. And also, the 'isql' tool is linked against libodbc.so library. And now. Lets look at one example API function now, e.g. SQLAllocConnect() (but the same problem may be with any other API function). This function is called directly from 'isql' somewhere - and on that place, there desired to call the SQLAllocConnect() function provided by libodbc.so. The 'isql' loads plugins through libodbc.so. The psqlodbcw.so plugin does not provide SQLAllocConnect(). But during loading of any plugin, libodbc.so tries to search for possible existence of that function inside the plugin. According to documented dlsym() behavior - if the SQLAllocConntect() function is not defined in the plugin, it tries to look more deeply in plugin's dependencies.. and it is successful ~> because one of the dependencies is: "libodbc.so". But because libodbc.so don't wan't to use that "internal" method - only if the method was defined in plugin directly, unixODBC guys prepared very ugly workaround for that situation - they try to "blacklist" addresses of internal methods into template_func[] array for later comparison with return of lt_dlsym() output. Now we have the problem: That template_func array is initialized (at least) on s390x differently than on other platforms - the template_func[0].dm_func is *CORRECTLY* initialized to the "internal" SQLAllocConntect() function address. But note! It is different address than also *CORRECT* address returned from lt_dlsym() call later - so detecting based on template_func[] is not successful (actually, the function is located in memory at least twice). You can look into unixODBC into the file DriverManager/SQLConnect.c: 582 /* 583 * structure to contain the loaded lib entry points 584 */ 585 586 static struct driver_func template_func[] = 587 { 588 /* 00 */ { SQL_API_SQLALLOCCONNECT, "SQLAllocConnect", (void*)SQLAllocConnect }, That part of template_func[0] is initialized to the same SQLAllocConnect address as it is in 'isql'. Actually, if there was _no_ direct call to SQLAllocConnect in 'isql' code, linker on s390 would assign to that place the expected address (the same which would be later returned by dlsym()). ---- I don't know yet whether the linker behavior is correct or not (whether such linker behavioral expectations are ok), I'll observe it in next few days and file a bug against s390x linker. Also, I'll try to ask for better unixODBC api later, probably. But firstly asking here: is that linkage against libodbc.so needed? I can see that the blacklisting ugly hack in unixODBC is there just because of plugins which are linked back against unixODBC. Pavel
Hi Pavel, (2013/11/12 2:50), Pavel Raiskup wrote: > Hello all! > > Long story short: Is there a need to link psqlodbcw.so plugin against > libodbc.so? Principal problem: That library provides ABI for applications > - not for plugins; at least it seem to be like that. > > I tried to remove this linking by following tweak: > > | diff --git a/configure.ac b/configure.ac > | index 5fb401e..5d75db3 100644 > | --- a/configure.ac > | +++ b/configure.ac > | @@ -65,7 +65,7 @@ fi > | # ODBC include and library > | # > | > | -if test "$ODBC_CONFIG" != ""; then > | +if false; then > | if test "$with_iodbc" != no; then > | ODBC_INCLUDE=`${ODBC_CONFIG} --cflags` > | CPPFLAGS="$CPPFLAGS ${ODBC_INCLUDE}" > > And both unixODBC and psqlodbcw.so communicates with each other correctly > at the first sight. Seems -lodbcinst is really necessary not -lodbc. Could you please try the attached patch for the current git? regards, Hiroshi Inoue
Attachment
> Seems -lodbcinst is really necessary not -lodbc. > Could you please try the attached patch for the current git? Hi, thanks a lot for looking at it! The patch works (I have lost access to s390x for a moment, but testing on that arch is not needed iiuic). That means hand testing via 'isql' works OK on x86_64. Also ldd is fine: $ ldd ./.libs/psqlodbcw.so | grep libodbc libodbcinst.so.2 => /lib64/libodbcinst.so.2 (0x00007fbf039b3000) But looking at configure.ac - we should probably fix the iodbc case.. but I'm not sure about that. Just a little update to this issue: I tried other architectures I am able to test and the problematic architectures seem to be at least armv7hl, s390x, s390. Thanks, Pavel
> But looking at configure.ac - we should probably fix the iodbc case.. but > I'm not sure about that. Please scratch that ^^ sentence. I have looked badly at the code - your patch should fix also iodbc case - so it is OK from my POV, thanks. (Tested also on s390x).
[+cc back psqlodbc] [+cc unixODBC] On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote: > On 11/11/13 17:50, Pavel Raiskup wrote: > > Hello all! > > > > Long story short: Is there a need to link psqlodbcw.so plugin against > > libodbc.so? Principal problem: That library provides ABI for applications > > - not for plugins; at least it seem to be like that. > > > > I tried to remove this linking by following tweak: > > Hi, > > As you have noticed, there is code in the DM to try and avoid what you > describe. But AFAIK, a driver would normally link against -lodbcinst to > gain access to the ini functions SQLGetPrivateProfileString and so on, > no need to link to the driver manager (-lodbc) that is for application > land use. Hello Nick, thanks for looking at the problem from unixODBC perspective! Would not there be better rather block loading badly linked plugins rather than just try to expect that the linking is done the same way on all arches? I attached possible solution. Pavel
Attachment
> I attached possible solution. I removed too much code in the previous patch. Attached once again.
Attachment
On 13/11/13 10:41, Pavel Raiskup wrote: > [+cc back psqlodbc] > [+cc unixODBC] > > On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote: >> On 11/11/13 17:50, Pavel Raiskup wrote: >>> Hello all! >>> >>> Long story short: Is there a need to link psqlodbcw.so plugin against >>> libodbc.so? Principal problem: That library provides ABI for applications >>> - not for plugins; at least it seem to be like that. >>> >>> I tried to remove this linking by following tweak: >> Hi, >> >> As you have noticed, there is code in the DM to try and avoid what you >> describe. But AFAIK, a driver would normally link against -lodbcinst to >> gain access to the ini functions SQLGetPrivateProfileString and so on, >> no need to link to the driver manager (-lodbc) that is for application >> land use. > Hello Nick, thanks for looking at the problem from unixODBC perspective! > > Would not there be better rather block loading badly linked plugins rather > than just try to expect that the linking is done the same way on all arches? > I attached possible solution. > > Pavel Yes, I see your solution. Possible though that the driver is linked against libiodbc.so or other non unixODBC version of ODBC driver manager and so has entry point for SQLAllocStmt (for example) that is not in the driver (as in this case). There are also situations I have seen where a driver decides to call itself (say SQLAllocStmt() in the driver makes a call to SQLAllocHandle() expecting the driver version) but finds the driver manager entry point. Anyway you have the good solution for your driver in linking against libodbcinst -- Nick Gorham
On 13/11/13 10:41, Pavel Raiskup wrote: > [+cc back psqlodbc] > [+cc unixODBC] > > On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote: >> On 11/11/13 17:50, Pavel Raiskup wrote: >>> Hello all! >>> >>> Long story short: Is there a need to link psqlodbcw.so plugin against >>> libodbc.so? Principal problem: That library provides ABI for applications >>> - not for plugins; at least it seem to be like that. >>> >>> I tried to remove this linking by following tweak: >> Hi, >> >> As you have noticed, there is code in the DM to try and avoid what you >> describe. But AFAIK, a driver would normally link against -lodbcinst to >> gain access to the ini functions SQLGetPrivateProfileString and so on, >> no need to link to the driver manager (-lodbc) that is for application >> land use. > Hello Nick, thanks for looking at the problem from unixODBC perspective! > > Would not there be better rather block loading badly linked plugins rather > than just try to expect that the linking is done the same way on all arches? > I attached possible solution. > > Pavel Yes, I see your solution. Possible though that the driver is linked against libiodbc.so or other non unixODBC version of ODBC driver manager and so has entry point for SQLAllocStmt (for example) that is not in the driver (as in this case). There are also situations I have seen where a driver decides to call itself (say SQLAllocStmt() in the driver makes a call to SQLAllocHandle() expecting the driver version) but finds the driver manager entry point. Anyway you have the good solution for your driver in linking against libodbcinst -- Nick Gorham