Re: PG v12.2 - Setting jit_above_cost is causing the server to crash - Mailing list pgsql-hackers

From Sandeep Thakkar
Subject Re: PG v12.2 - Setting jit_above_cost is causing the server to crash
Date
Msg-id CANFyU96Lhhter1a6cUzov-tFv8aevz5AeXSRH1Jgwa41N8owxA@mail.gmail.com
Whole thread Raw
In response to Re: PG v12.2 - Setting jit_above_cost is causing the server to crash  (Dave Page <dpage@pgadmin.org>)
List pgsql-hackers
Hi,

On Thu, Feb 27, 2020 at 6:23 PM Dave Page <dpage@pgadmin.org> wrote:
Hi

On Thu, Feb 27, 2020 at 12:41 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> writes:
> On Mon, Feb 24, 2020 at 12:46 PM Andres Freund <andres@anarazel.de> wrote:
>> This isn't reproducible here. Are you sure that you're running on a
>> clean installation?

> Yes I did a fresh installation using installer provided here -
> https://www.enterprisedb.com/downloads/postgresql

There is apparently something wrong with the JIT stuff in EDB's 12.2
build for macOS.  At least, that's the conclusion I came to after
off-list discussion with the submitter of bug #16264, which has pretty
much exactly this symptom (especially if you're seeing "signal 9"
reports in the postmaster log).  For him, either disabling JIT or
reverting to 12.1 made it go away.

We've been looking into this;

Apple started a notarisation process some time ago, designed to mark their applications as conforming to various security requirements, but prior to Catalina it was essentially optional. When Catalina was released, they made notarisation for distributed software a requirement, but had the process issue warnings for non-compliance. As-of the end of January, those warnings became hard errors, so now our packages must be notarised, and for that to happen, must be hardened by linking with a special runtime and having securely time stamped signatures on every binary before being checked and notarised as such by Apple. Without that, users would have to disable security features on their systems before they could run our software.

Our packages are being successfully notarised at the moment, because that's essentially done through a static analysis. We can (and have) added what Apple call an entitlement in test builds which essentially puts a flag in the notarisation for the product that declares that it will do JIT operations, however, it seems that this alone is not enough and that in addition to the entitlement, we also need to include the MAP_JIT flag in mmap() calls. See https://developer.apple.com/documentation/security/hardened_runtime and https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit

We're working on trying to test a patch for that at the moment.
 
We have fixed the issue. To explain in brief, It was related to the hardened runtime. Hardening the runtime can produce such issues, and therefore Apple provides the runtime exceptions. We were previously using an entitlement "com.apple.security.cs.disable-library-validation" for the app bundle and then tried adding "com.apple.security.cs.allow-unsigned-executable-memory" but still the query would crash the server process when JIT is enabled. Later we applied these entitlements to the PG binaries (postgres, pg_ctl and others) and the bundles (llvmjit.so and others) which actually resolved the problem.

The updates will be released in a couple of days.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Sandeep Thakkar


pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: [PATCH] Skip llvm bytecode generation if LLVM is missing
Next
From: Pavel Stehule
Date:
Subject: Re: plan cache overhead on plpgsql expression