Thread: conflicting libraries at runtime

I know this is a bit off-topic, but I was hoping someone can point me in 
the right direction. I've been spinning my wheels on this for a while now.

I'm having trouble with PL/R giving me a SIGSEGV *only* on Red Hat 9. 
I've isolated it (at least the symptom) down to this: on RH9, a call to 
re_compile_fastmap() uses /lib/tls/libc.so.6 instead of the 
compiled-into-R function with the same name. The /lib/tls/libc.so.6 
version of the function calls re_compile_fastmap_iter() which then 
generates the SIGSEGV. I have tested a very simple standalone app, on 
the same box, also linked to the same libR.so, and it uses R's builtin 
re_compile_fastmap() (which has no re_compile_fastmap_iter() function). 
The standalone app works fine. These calls happen during the early 
initialization process of R.

R itself works fine and passes all of it's regression tests.

I've tested *exactly* the same PL/R code on RH8 and RH7.3 boxen with no 
problems. gdb shows that on those machines the libR version of 
re_compile_fastmap() is used, just like the standalone app on RH9.

Any ideas? I'd be happy to provide more info (off list) if anyone wants it.

Thanks,

Joe

Joe Conway <mail@joeconway.com> writes:
> I'm having trouble with PL/R giving me a SIGSEGV *only* on Red Hat 9. 
> I've isolated it (at least the symptom) down to this: on RH9, a call to 
> re_compile_fastmap() uses /lib/tls/libc.so.6 instead of the 
> compiled-into-R function with the same name.

I tossed this up to an internal Red Hat mailing list, and got the
following reply from Ulrich Drepper:

> If the problem is really that, against the expectations, the dynamic
> linker picks the implementation in libc.so as opposed to the program, use
>   env LD_DEBUG=all LD_DEBUG_OUTPUT=somefile normal_program args...
> to see a lot of output on the name resolution process.  Search for the
> symbol you're looking for in this output file.

We may have to go back to Ulrich to decode what the output tells us,
but it's something to try anyway ...
        regards, tom lane

Joe Conway writes:

> I'm having trouble with PL/R giving me a SIGSEGV *only* on Red Hat 9.
> I've isolated it (at least the symptom) down to this: on RH9, a call to
> re_compile_fastmap() uses /lib/tls/libc.so.6 instead of the
> compiled-into-R function with the same name.

The linker flag -Bsymbolic can help in these situations (at least on ELF
systems).  In the long term you need to fix the function names, though.

-- 
Peter Eisentraut   peter_e@gmx.net

Tom Lane wrote:
>>If the problem is really that, against the expectations, the dynamic
>>linker picks the implementation in libc.so as opposed to the program, use
>>  env LD_DEBUG=all LD_DEBUG_OUTPUT=somefile normal_program args...
>>to see a lot of output on the name resolution process.  Search for the
>>symbol you're looking for in this output file.
>
> We may have to go back to Ulrich to decode what the output tells us,
> but it's something to try anyway ...
>

Thanks! I've attached the relevant portions from a Red Hat 8 and a Red
Hat 9 machine. There are two included symbols for the contrast:

   - symbol=fft_factor gets resolved correctly to
     /usr/local/lib/R/bin/libR.so in both cases
   - symbol=re_compile_fastmap is correctly resolved on RH8, and
     incorrectly (at least for my needs) on RH9

But it seems that libc.so.6 on both RH8 and RH9 have the symbol
re_compile_fastmap (see snipped line from nm output in attachment), so I
still don't understand why in RH9 the libc symbol is used, and in RH8
the one from libR.

Thanks,

Joe

env LD_DEBUG=all LD_DEBUG_OUTPUT=/home/postgres/dbg.out pg_ctl start

rh8 good:
     19097:    symbol=fft_factor;  lookup in file=postgres: postgres regression [local] SELECT
     19097:    symbol=fft_factor;  lookup in file=/usr/lib/libz.so.1
     19097:    symbol=fft_factor;  lookup in file=/usr/lib/libreadline.so.4
     19097:    symbol=fft_factor;  lookup in file=/lib/libtermcap.so.2
     19097:    symbol=fft_factor;  lookup in file=/lib/libcrypt.so.1
     19097:    symbol=fft_factor;  lookup in file=/lib/libresolv.so.2
     19097:    symbol=fft_factor;  lookup in file=/lib/libnsl.so.1
     19097:    symbol=fft_factor;  lookup in file=/lib/libdl.so.2
     19097:    symbol=fft_factor;  lookup in file=/lib/i686/libm.so.6
     19097:    symbol=fft_factor;  lookup in file=/lib/i686/libc.so.6
     19097:    symbol=fft_factor;  lookup in file=/lib/ld-linux.so.2
     19097:    symbol=fft_factor;  lookup in file=/usr/local/pgsql/lib/plr.so
     19097:    symbol=fft_factor;  lookup in file=/usr/local/lib/R/bin/libR.so
     19097:    binding file /usr/local/lib/R/bin/libR.so to /usr/local/lib/R/bin/libR.so: normal symbol `fft_factor'
     19097:    symbol=re_compile_fastmap;  lookup in file=postgres: postgres regression [local] SELECT
     19097:    symbol=re_compile_fastmap;  lookup in file=/usr/lib/libz.so.1
     19097:    symbol=re_compile_fastmap;  lookup in file=/usr/lib/libreadline.so.4
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/libtermcap.so.2
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/libcrypt.so.1
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/libresolv.so.2
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/libnsl.so.1
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/libdl.so.2
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/i686/libm.so.6
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/i686/libc.so.6
     19097:    symbol=re_compile_fastmap;  lookup in file=/lib/ld-linux.so.2
     19097:    symbol=re_compile_fastmap;  lookup in file=/usr/local/pgsql/lib/plr.so
     19097:    symbol=re_compile_fastmap;  lookup in file=/usr/local/lib/R/bin/libR.so
     19097:    binding file /usr/local/lib/R/bin/libR.so to /usr/local/lib/R/bin/libR.so: normal symbol
`re_compile_fastmap'

     nm /lib/i686/libc.so.6
     000b7dc0 W re_compile_fastmap

rh9 bad:
      2074:    symbol=fft_factor;  lookup in file=postgres: postgres regression [local] SELECT
      2074:    symbol=fft_factor;  lookup in file=/usr/lib/libz.so.1
      2074:    symbol=fft_factor;  lookup in file=/usr/lib/libreadline.so.4
      2074:    symbol=fft_factor;  lookup in file=/lib/libtermcap.so.2
      2074:    symbol=fft_factor;  lookup in file=/lib/libcrypt.so.1
      2074:    symbol=fft_factor;  lookup in file=/lib/libresolv.so.2
      2074:    symbol=fft_factor;  lookup in file=/lib/libnsl.so.1
      2074:    symbol=fft_factor;  lookup in file=/lib/libdl.so.2
      2074:    symbol=fft_factor;  lookup in file=/lib/tls/libm.so.6
      2074:    symbol=fft_factor;  lookup in file=/lib/tls/libc.so.6
      2074:    symbol=fft_factor;  lookup in file=/lib/ld-linux.so.2
      2074:    symbol=fft_factor;  lookup in file=/usr/lib/postgresql/plr.so
      2074:    symbol=fft_factor;  lookup in file=/usr/local/lib/R/bin/libR.so
      2074:    binding file /usr/local/lib/R/bin/libR.so to /usr/local/lib/R/bin/libR.so: normal symbol `fft_factor'
      2074:    symbol=re_compile_fastmap;  lookup in file=postgres: postgres regression [local] SELECT
      2074:    symbol=re_compile_fastmap;  lookup in file=/usr/lib/libz.so.1
      2074:    symbol=re_compile_fastmap;  lookup in file=/usr/lib/libreadline.so.4
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/libtermcap.so.2
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/libcrypt.so.1
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/libresolv.so.2
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/libnsl.so.1
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/libdl.so.2
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/tls/libm.so.6
      2074:    symbol=re_compile_fastmap;  lookup in file=/lib/tls/libc.so.6
      2074:    binding file /usr/local/lib/R/bin/libR.so to /lib/tls/libc.so.6: normal symbol `re_compile_fastmap'

      nm /lib/tls/libc.so.6
      420b6450 W re_compile_fastmap

Peter Eisentraut wrote:
> Joe Conway writes:
>>I'm having trouble with PL/R giving me a SIGSEGV *only* on Red Hat 9.
>>I've isolated it (at least the symptom) down to this: on RH9, a call to
>>re_compile_fastmap() uses /lib/tls/libc.so.6 instead of the
>>compiled-into-R function with the same name.
> 
> The linker flag -Bsymbolic can help in these situations (at least on ELF
> systems).  In the long term you need to fix the function names, though.

Thanks, Peter. I'll give that a try.

The function name is part of R itself, not PL/R, so I guess I'll contact 
the R developers with this recommendation.

Thanks,

Joe