Thread: a curious case of force_parallel_mode = on with jit'ing

a curious case of force_parallel_mode = on with jit'ing

From
Amit Langote
Date:
Hi,

Has anybody seen this:

$ psql regression
Timing is on.
psql (14devel)
Type "help" for help.

regression=# set force_parallel_mode to on;
SET
Time: 0.888 ms
regression=# set jit to on;
SET
Time: 0.487 ms
regression=# set jit_above_cost to 1;
SET
Time: 0.476 ms
regression=# SELECT p1.f1, p2.f1, p1.f1 * p2.f1 FROM POINT_TBL p1,
POINT_TBL p2 WHERE p1.f1[0] < 1;
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process
exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
ERROR:  value out of range: underflow
CONTEXT:  parallel worker
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
Time: 670.801 ms
!?> \q

(gdb) bt
#0  0x00007f57ac6508cd in std::_Function_handler<void (unsigned long,
llvm::object::ObjectFile const&),
llvm::OrcCBindingsStack::OrcCBindingsStack(llvm::TargetMachine&,
std::function<std::unique_ptr<llvm::orc::IndirectStubsManager,
std::default_delete<llvm::orc::IndirectStubsManager> >
()>)::{lambda(unsigned long, llvm::object::ObjectFile
const&)#3}>::_M_invoke(std::_Any_data const&, unsigned long,
llvm::object::ObjectFile const&) () from
/opt/rh/llvm-toolset-7.0/root/usr/lib64/libLLVM-7.so
#1  0x00007f57ac652578 in
llvm::orc::RTDyldObjectLinkingLayer::ConcreteLinkedObject<std::shared_ptr<llvm::RuntimeDyld::MemoryManager>
>::~ConcreteLinkedObject() () from
/opt/rh/llvm-toolset-7.0/root/usr/lib64/libLLVM-7.so
#2  0x00007f57ac6527aa in std::_Rb_tree<unsigned long,
std::pair<unsigned long const,
std::unique_ptr<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject,
std::default_delete<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject>
> >, std::_Select1st<std::pair<unsigned long const,
std::unique_ptr<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject,
std::default_delete<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject>
> > >, std::less<unsigned long>, std::allocator<std::pair<unsigned
long const, std::unique_ptr<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject,
std::default_delete<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject>
> > > >::_M_erase(std::_Rb_tree_node<std::pair<unsigned long const,
std::unique_ptr<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject,
std::default_delete<llvm::orc::RTDyldObjectLinkingLayerBase::LinkedObject>
> > >*) () from /opt/rh/llvm-toolset-7.0/root/usr/lib64/libLLVM-7.so
#3  0x00007f57ac65ec91 in
llvm::OrcCBindingsStack::~OrcCBindingsStack() () from
/opt/rh/llvm-toolset-7.0/root/usr/lib64/libLLVM-7.so
#4  0x00007f57ac65efaa in LLVMOrcDisposeInstance () from
/opt/rh/llvm-toolset-7.0/root/usr/lib64/libLLVM-7.so
#5  0x00007f57ae62d7bf in llvm_shutdown (code=1, arg=0) at llvmjit.c:926
#6  0x0000000000916d00 in proc_exit_prepare (code=1) at ipc.c:209
#7  0x0000000000916bdb in proc_exit (code=1) at ipc.c:107
#8  0x000000000087e8d6 in StartBackgroundWorker () at bgworker.c:832
#9  0x0000000000892fa7 in do_start_bgworker (rw=0x2dae8a0) at postmaster.c:5833
#10 0x0000000000893355 in maybe_start_bgworkers () at postmaster.c:6058
#11 0x0000000000892390 in sigusr1_handler (postgres_signal_arg=10) at
postmaster.c:5215
#12 <signal handler called>
#13 0x00007f57d4e20933 in __select_nocancel () from /lib64/libc.so.6
#14 0x000000000088e00e in ServerLoop () at postmaster.c:1694
#15 0x000000000088d9fd in PostmasterMain (argc=5, argv=0x2d863f0) at
postmaster.c:1402
#16 0x0000000000791197 in main (argc=5, argv=0x2d863f0) at main.c:209
(gdb)

I can make this happen with PG > v12.  Maybe there's something wrong
with my LLVM installation but just thought to ask here just in case.

-- 
Amit Langote
EDB: http://www.enterprisedb.com



Re: a curious case of force_parallel_mode = on with jit'ing

From
Tom Lane
Date:
Amit Langote <amitlangote09@gmail.com> writes:
> Has anybody seen this:

Works for me on HEAD (using RHEL8.3, gcc 8.3.1, LLVM 10.0.1).

            regards, tom lane



Re: a curious case of force_parallel_mode = on with jit'ing

From
Amit Langote
Date:
On Thu, Feb 4, 2021 at 1:08 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Amit Langote <amitlangote09@gmail.com> writes:
> > Has anybody seen this:
>
> Works for me on HEAD (using RHEL8.3, gcc 8.3.1, LLVM 10.0.1).

Thanks for checking.  Must be my LLVM setup I guess:

$ llvm-config --version
7.0.1
$ cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
$ gcc --version
gcc (GCC) 9.1.1 20190605 (Red Hat 9.1.1-2)

-- 
Amit Langote
EDB: http://www.enterprisedb.com



Re: a curious case of force_parallel_mode = on with jit'ing

From
Tom Lane
Date:
Amit Langote <amitlangote09@gmail.com> writes:
> On Thu, Feb 4, 2021 at 1:08 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Works for me on HEAD (using RHEL8.3, gcc 8.3.1, LLVM 10.0.1).

> Thanks for checking.  Must be my LLVM setup I guess:

> $ llvm-config --version
> 7.0.1
> $ cat /etc/redhat-release
> CentOS Linux release 7.7.1908 (Core)
> $ gcc --version
> gcc (GCC) 9.1.1 20190605 (Red Hat 9.1.1-2)

Hmmm ... seems like an odd combination to have a newer gcc and an
older LLVM than what RHEL8 is shipping.  Is this really the current
recommendation on CentOS 7?

            regards, tom lane



Re: a curious case of force_parallel_mode = on with jit'ing

From
Amit Langote
Date:
On Thu, Feb 4, 2021 at 1:41 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Amit Langote <amitlangote09@gmail.com> writes:
> > On Thu, Feb 4, 2021 at 1:08 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Works for me on HEAD (using RHEL8.3, gcc 8.3.1, LLVM 10.0.1).
>
> > Thanks for checking.  Must be my LLVM setup I guess:
>
> > $ llvm-config --version
> > 7.0.1
> > $ cat /etc/redhat-release
> > CentOS Linux release 7.7.1908 (Core)
> > $ gcc --version
> > gcc (GCC) 9.1.1 20190605 (Red Hat 9.1.1-2)
>
> Hmmm ... seems like an odd combination to have a newer gcc and an
> older LLVM than what RHEL8 is shipping.  Is this really the current
> recommendation on CentOS 7?

Not an official combination.  At some point last year I decided to
install a more modern gcc than what CentOS 7 officially provides and
ended up getting them through a Software Collections (scl) package
called devtoolset-9.

-- 
Amit Langote
EDB: http://www.enterprisedb.com