Thread: BUG #15969: PG12 b3 with LLVM JIT can't load library llvmjit.so: unresolvd symbol llvm::fatal_error_handler

The following bug has been logged on the website:

Bug reference:      15969
Logged by:          Sylvie Empereur
Email address:      sylvie.empereur-mot@bull.net
PostgreSQL version: 12beta3
Operating system:   Linux RedHat 7.6
Description:

SW and HW Environment:
-    LINUX RedHat Entreprise 7.6   (ppc64le on Power9 processors)
-    GCC 4.8.5  (included in RHEL7.6)
-    make 3.8.2
-       LLVM8 and CLANG8
-    PostgreSQL 12 beta3 built with support for LLVM JIT based compilation 
-    Doc PostgreSQL (chap31 JIT Compilation) indicates script configure, to be
run with parameter LLVM_CONFIG.  
-       PG12 script configure requires parameter CLANG also.
configure --without-readline --with-llvm
LLVM_CONFIG='/home/postgres/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/bin/llvm-config'
CLANG='/home/postgres/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/bin/clang'
make clean
make -j40 world
make check

Error Symptom:   
-    make check  step, shows 26 identical failures from total of 193 autotests
:
ERROR:  could not load library
"/nvme/postgresql-12beta3/tmp_install/usr/local/pgsql/lib/llvmjit.so":
/nvme/postgresql-12beta3/tmp_install/usr/local/pgsql/lib/llvmjit.so:
undefined symbol: _ZNSt3_V214error_categoryD2Ev
-    The symbol C++ library  llvm::fatal_error_handler is defined in CLANG
library libclang.so
-    Symbol resolution shows up when the shared lib llvmjit.so tries to resolve
symbol llvm::fatal_error_handler, defined in the shared library
libclang.so
-    Symbol table entries in CLANG/LLVM library libclang.so
$ objdump –t
$HOME/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/lib/libclang.so |grep -i
fatal_error_handler
/home/postgresql-12beta2/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/lib/libclang.so:
 file format elf64-powerpcle
SYMBOL TABLE: 
00000000005ae150 l     F .text  0000000000000048   0x60
fatal_error_handler(void*, std::string const&, bool)
000000000220fe30 l     F .text  00000000000000cc   0x60
llvm::install_fatal_error_handler(void (*)(void*, std::string const&, bool),
void*)
The flag characters as follows:     "l"   =>    The symbol is a local (l)

Hi,

On 2019-08-20 16:41:06 +0000, PG Bug reporting form wrote:
> The following bug has been logged on the website:
> 
> Bug reference:      15969
> Logged by:          Sylvie Empereur
> Email address:      sylvie.empereur-mot@bull.net
> PostgreSQL version: 12beta3
> Operating system:   Linux RedHat 7.6
> Description:        
> 
> SW and HW Environment:
> -    LINUX RedHat Entreprise 7.6   (ppc64le on Power9 processors)
> -    GCC 4.8.5  (included in RHEL7.6)
> -    make 3.8.2
> -       LLVM8 and CLANG8
> -    PostgreSQL 12 beta3 built with support for LLVM JIT based compilation 
> -    Doc PostgreSQL (chap31 JIT Compilation) indicates script configure, to be
> run with parameter LLVM_CONFIG.  
> -       PG12 script configure requires parameter CLANG also.
> configure --without-readline --with-llvm
> LLVM_CONFIG='/home/postgres/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/bin/llvm-config'
> CLANG='/home/postgres/clang+llvm-8.0.1-powerpc64le-linux-rhel-7.4/bin/clang'
> make clean
> make -j40 world
> make check

I suspect the problem here might be that you're building against an LLVM
version which is not in dynamic linker's path. So when postgres loads
llvmjit.so the dynamic linker will find the globally installed version
of the llvm libraries, which is not compatible with the self installed
version.  If that's indeed the problem, you can either
1) adapt your dynamic linker configuration to look for shared libraries
   inside the appropriate directory, e.g. by setting LD_LIBRARY_PATH, or
   by configuring things more permanently in /etc/ld.so.conf[.d] (don't
   forget to run ldconfig in that case)
2) Add a flag to postgres compilation changing the library lookups,
   using rpath. E.g. I'm using something like
   LDFLAGS='-Wl,-rpath,/home/andres/build/llvm/opt/install/lib'


> Error Symptom:   
> -    make check  step, shows 26 identical failures from total of 193 autotests
> :
> ERROR:  could not load library
> "/nvme/postgresql-12beta3/tmp_install/usr/local/pgsql/lib/llvmjit.so":
> /nvme/postgresql-12beta3/tmp_install/usr/local/pgsql/lib/llvmjit.so:
> undefined symbol: _ZNSt3_V214error_categoryD2Ev
> -    The symbol C++ library  llvm::fatal_error_handler is defined in CLANG
> library libclang.so

I don't think libclang should play any role here, because postgres's jit
doesn't use libclang at runtime (you can argue it uses it via the clang
binary at compile time, but that wouldn't cause errors like this).

Greetings,

Andres Freund