Thread: Build fails with different versions of clang and LLVM
Hi! I tried compiling with --with-llvm on my laptop, and got this: > $ make -s > All of PostgreSQL successfully made. Ready to install. > $ make -s install > Invalid Summary Block: version expected > error: can't create ModuleSummaryIndexObjectFile for buffer: Corrupted bitcode > #0 0x00007fbe98032085 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x707085) > #1 0x00007fbe9803023e llvm::sys::RunSignalHandlers() (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x70523e) > #2 0x00007fbe98030362 (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x705362) > #3 0x00007fbe9771f160 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12160) > #4 0x00007fbe985cfba4 (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0xca4ba4) > #5 0x00007fbe985e3318 llvm::WriteIndexToFile(llvm::ModuleSummaryIndex const&, llvm::raw_ostream&, std::map<std::__cxx11::basic_string<char,std::char_traits<char>, std::allocator<char> >, std::map<unsigned long, llvm::GlobalValueSummary*,std::less<unsigned long>, std::allocator<std::pair<unsigned long const, llvm::GlobalValueSummary*>> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::map<unsignedlong, llvm::GlobalValueSummary*, std::less<unsigned long>, std::allocator<std::pair<unsigned long const,llvm::GlobalValueSummary*> > > > > >*) (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0xcb8318) > #6 0x000055d1120f4ac7 (/usr/lib/llvm-3.9/bin/llvm-lto+0x19ac7) > #7 0x000055d1120f5f95 (/usr/lib/llvm-3.9/bin/llvm-lto+0x1af95) > #8 0x000055d1120ea0f0 _init (/usr/lib/llvm-3.9/bin/llvm-lto+0xf0f0) > #9 0x00007fbe96a96f2a __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20f2a) > #10 0x000055d1120ebe8a _init (/usr/lib/llvm-3.9/bin/llvm-lto+0x10e8a) > Stack dump: > 0. Program arguments: /usr/lib/llvm-3.9/bin/llvm-lto -thinlto -thinlto-action=thinlink -o postgres.index.bc postgres/access/brin/brin.bcpostgres/access/brin/brin_pageops.bc [LOT OF FILES REMOVED TO KEEP THIS SHORT] postgres/utils/fmgrtab.bcpostgres/jit/jit.bc > Segmentation fault > Makefile:246: recipe for target 'install-postgres-bitcode' failed > make[2]: *** [install-postgres-bitcode] Error 139 > Makefile:42: recipe for target 'install-backend-recurse' failed > make[1]: *** [install-backend-recurse] Error 2 > GNUmakefile:11: recipe for target 'install-src-recurse' failed > make: *** [install-src-recurse] Error 2 I think this is because I have a funny combination of clang and LLVM: > $ clang --version > clang version 3.8.1-24 (tags/RELEASE_381/final) > Target: x86_64-pc-linux-gnu > Thread model: posix > InstalledDir: /usr/bin > $ llvm-config --version > bash: llvm-config: command not found > $ llvm-config-3.9 --version > 3.9.1 So, I have LLVM 3.9 installed, but no clang 3.9. Only clang 3.8. The autoconf macro in llvm.m4 says: > # Could check clang version, but it doesn't seem that > # important. Systems with a new enough LLVM version are usually > # going to have a decent clang version too. It's also not entirely > # clear what the minimum version is. In light of the above error, I think we should do more. Apparently LLVM 3.9 cannot read bitcode files generated by clang 3.8, so we should at least check that clang version is 3.9 or above. Googling around, the LLVM bitcode format is supposed to be compatible across versions, but I'm not sure I trust that. Perhaps we should check that the LLVM and clang versions match? Instead of searching for any 'clang' program with PGAC_PATH_PROGS, perhaps we should always use "`lvm-config-3.9 --bindir`/clang", throwing an error if it doesn't exist. BTW, I'm surprised that llvm-lto is invoked in the "make install" stage. I would expect it to be done by plain "make" already, and "make install" would just copy the files to the right places. - Heikki
On Monday, April 23, 2018 10:33:04 AM CEST Heikki Linnakangas wrote: > Hi! > > I tried compiling with --with-llvm on my laptop, and got this: > > $ make -s > > All of PostgreSQL successfully made. Ready to install. > > $ make -s install > > Invalid Summary Block: version expected > > error: can't create ModuleSummaryIndexObjectFile for buffer: Corrupted > > bitcode #0 0x00007fbe98032085 > > llvm::sys::PrintStackTrace(llvm::raw_ostream&) > > (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x707085) #1 > > 0x00007fbe9803023e llvm::sys::RunSignalHandlers() > > (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x70523e) #2 > > 0x00007fbe98030362 > > (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0x705362) #3 > > 0x00007fbe9771f160 __restore_rt > > (/lib/x86_64-linux-gnu/libpthread.so.0+0x12160) #4 0x00007fbe985cfba4 > > (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0xca4ba4) #5 > > 0x00007fbe985e3318 llvm::WriteIndexToFile(llvm::ModuleSummaryIndex > > const&, llvm::raw_ostream&, std::map<std::__cxx11::basic_string<char, > > std::char_traits<char>, std::allocator<char> >, std::map<unsigned long, > > llvm::GlobalValueSummary*, std::less<unsigned long>, > > std::allocator<std::pair<unsigned long const, llvm::GlobalValueSummary*> > > > >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, > > std::allocator<char> > >, > > std::allocator<std::pair<std::__cxx11::basic_string<char, > > std::char_traits<char>, std::allocator<char> > const, std::map<unsigned > > long, llvm::GlobalValueSummary*, std::less<unsigned long>, > > std::allocator<std::pair<unsigned long const, llvm::GlobalValueSummary*> > > > > > > >*) (/usr/lib/llvm-3.9/bin/../lib/libLLVM-3.9.so.1+0xcb8318) #6 > > 0x000055d1120f4ac7 (/usr/lib/llvm-3.9/bin/llvm-lto+0x19ac7) > > #7 0x000055d1120f5f95 (/usr/lib/llvm-3.9/bin/llvm-lto+0x1af95) > > #8 0x000055d1120ea0f0 _init (/usr/lib/llvm-3.9/bin/llvm-lto+0xf0f0) > > #9 0x00007fbe96a96f2a __libc_start_main > > (/lib/x86_64-linux-gnu/libc.so.6+0x20f2a) #10 0x000055d1120ebe8a _init > > (/usr/lib/llvm-3.9/bin/llvm-lto+0x10e8a) Stack dump: > > 0. Program arguments: /usr/lib/llvm-3.9/bin/llvm-lto -thinlto > > -thinlto-action=thinlink -o postgres.index.bc > > postgres/access/brin/brin.bc postgres/access/brin/brin_pageops.bc [LOT OF > > FILES REMOVED TO KEEP THIS SHORT] postgres/utils/fmgrtab.bc > > postgres/jit/jit.bc Segmentation fault > > Makefile:246: recipe for target 'install-postgres-bitcode' failed > > make[2]: *** [install-postgres-bitcode] Error 139 > > Makefile:42: recipe for target 'install-backend-recurse' failed > > make[1]: *** [install-backend-recurse] Error 2 > > GNUmakefile:11: recipe for target 'install-src-recurse' failed > > make: *** [install-src-recurse] Error 2 > > I think this is because I have a funny combination of clang and LLVM: > > $ clang --version > > clang version 3.8.1-24 (tags/RELEASE_381/final) > > Target: x86_64-pc-linux-gnu > > Thread model: posix > > InstalledDir: /usr/bin > > $ llvm-config --version > > bash: llvm-config: command not found > > $ llvm-config-3.9 --version > > 3.9.1 > > So, I have LLVM 3.9 installed, but no clang 3.9. Only clang 3.8. > > The autoconf macro in llvm.m4 says: > > # Could check clang version, but it doesn't seem that > > # important. Systems with a new enough LLVM version are usually > > # going to have a decent clang version too. It's also not entirely > > # clear what the minimum version is. > > In light of the above error, I think we should do more. Apparently LLVM > 3.9 cannot read bitcode files generated by clang 3.8, so we should at > least check that clang version is 3.9 or above. > > Googling around, the LLVM bitcode format is supposed to be compatible > across versions, but I'm not sure I trust that. Perhaps we should check > that the LLVM and clang versions match? Instead of searching for any > 'clang' program with PGAC_PATH_PROGS, perhaps we should always use > "`lvm-config-3.9 --bindir`/clang", throwing an error if it doesn't exist. > > BTW, I'm surprised that llvm-lto is invoked in the "make install" stage. > I would expect it to be done by plain "make" already, and "make install" > would just copy the files to the right places. Hi The bitcode format does not respect strict compatibility rules. LLVM and clang versions must match if we don't want any surprise. See for instance https://stackoverflow.com/questions/15836430/how-stable-is-the-llvm-assembly-language In your case, this should have worked, but I am not surprised at all that this failed, I had similar issues when testing 3.9/4.0 on the same system. I thought the build system already checked for version equality. Pierre
Hi, On 2018-04-23 04:33:04 -0400, Heikki Linnakangas wrote: > So, I have LLVM 3.9 installed, but no clang 3.9. Only clang 3.8. Any specific reason, or just happenstance? > Googling around, the LLVM bitcode format is supposed to be compatible across > versions, but I'm not sure I trust that. Perhaps we should check that the > LLVM and clang versions match? Instead of searching for any 'clang' program > with PGAC_PATH_PROGS, perhaps we should always use "`lvm-config-3.9 > --bindir`/clang", throwing an error if it doesn't exist. That'll commonly not exist I think. I kinda think we should just wait a bit till we've collected more experience.. > BTW, I'm surprised that llvm-lto is invoked in the "make install" stage. I > would expect it to be done by plain "make" already, and "make install" would > just copy the files to the right places. If you can come up with decent make foo, I'm open to changing that, but I couldn't come up with something neatly encapsulated... It's just an index over the individual files, so it didn't seem that crazy. Greetings, Andres Freund
On 23/04/18 18:28, Andres Freund wrote: > Hi, > > On 2018-04-23 04:33:04 -0400, Heikki Linnakangas wrote: >> So, I have LLVM 3.9 installed, but no clang 3.9. Only clang 3.8. > > Any specific reason, or just happenstance? Just happenstance. I had clang and llvm 3.8 installed previously. But PostgreSQL needs llvm 3.9, so I installed that, but I didn't think to install clang 3.9. To add to the story: I installed clang 3.9, but I still got the same error. I now had both clang 3.8 and 3.9 installed, but /usr/bin/clang still pointed to clang-3.8, so autoconf still picked that up. Only after removing clang-3.8 altogether, it started working. For the record, this is on a pretty normal Debian Stretch installation. - Heikki
Hi, On 2018-04-23 18:33:30 +0300, Heikki Linnakangas wrote: > To add to the story: I installed clang 3.9, but I still got the same error. > I now had both clang 3.8 and 3.9 installed, but /usr/bin/clang still pointed > to clang-3.8, so autoconf still picked that up. Only after removing > clang-3.8 altogether, it started working. And CLANG=clang-3.9 to autoconf didn't cure that? I've experimented a bit w/ this and there seem to be fewer cross-version issues in newer clang/llvm versions. Greetings, Andres Freund
On 23/04/18 18:35, Andres Freund wrote: > Hi, > > On 2018-04-23 18:33:30 +0300, Heikki Linnakangas wrote: >> To add to the story: I installed clang 3.9, but I still got the same error. >> I now had both clang 3.8 and 3.9 installed, but /usr/bin/clang still pointed >> to clang-3.8, so autoconf still picked that up. Only after removing >> clang-3.8 altogether, it started working. > > And CLANG=clang-3.9 to autoconf didn't cure that? Oh, I'm sure it would have, I didn't try that :-). > I've experimented a bit w/ this and there seem to be fewer cross-version > issues in newer clang/llvm versions. Good to hear. Nevertheless, I think we should try a bit harder to ensure that we pick the same version of clang and LLVM. Even if they're compatible, just seems more straightforward that way. And we now know there is an incompatibility between 3.8 and 3.9, at least. - Heikki