Thread: [MASSMAIL]macOS Ventura won't generate core dumps

[MASSMAIL]macOS Ventura won't generate core dumps

From
Robert Haas
Date:
At least not for me. According to various things I found on the
Internet, it's now required that you codesign your binaries and give
them an entitlement in order to generate core dumps:

https://nasa.github.io/trick/howto_guides/How-to-dump-core-file-on-MacOS.html

But according to previous on-list discussion, code-signing a binary
means that DYLD_* will be ignored:

http://postgr.es/m/920451.1634265967@sss.pgh.pa.us

Now, if DYLD_* is ignored, then our regression tests won't work
properly. But if core dumps are not enabled, then how am I supposed to
debug things that can only be debugged with a core dump?

Unless I'm missing something, this is positively rage-inducing. It's
reasonable, on Apple's part, to want to install more secure defaults,
but having no practical way of overriding the defaults is not
reasonable. And by "practical," I mean ideally (a) doesn't require a
macOS-specific patch to the PostgreSQL source repository but at least
(b) can be done somehow without breaking other important things that
also need to work.

Anyone have any ideas?

-- 
Robert Haas
EDB: http://www.enterprisedb.com



Re: macOS Ventura won't generate core dumps

From
Andres Freund
Date:
Hi,

On 2024-04-09 13:35:51 -0400, Robert Haas wrote:
> Now, if DYLD_* is ignored, then our regression tests won't work
> properly. But if core dumps are not enabled, then how am I supposed to
> debug things that can only be debugged with a core dump?

FWIW, I posted a patch a while back to make meson builds support relative
rpaths on some platforms, including macos. I.e. each library/binary finds the
location of the needed libraries relative to its own location. That gets rid
of the need to use DYLD_ at all, because the temporary install for the tests
can find the library location without a problem.

Perhaps it's worth picking that up again?

https://github.com/anarazel/postgres/tree/meson-rpath
https://github.com/anarazel/postgres/commit/46f1963fee7525c3cc3837ef8423cbf6cb08d10a

Greetings,

Andres Freund



Re: macOS Ventura won't generate core dumps

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> At least not for me. According to various things I found on the
> Internet, it's now required that you codesign your binaries and give
> them an entitlement in order to generate core dumps:

Still works for me, at least on machines where I have SIP turned
off.  Admittedly, Apple's busy making that a less and less desirable
choice.

> Unless I'm missing something, this is positively rage-inducing.

I don't disagree.

            regards, tom lane



Re: macOS Ventura won't generate core dumps

From
Robert Haas
Date:
On Tue, Apr 9, 2024 at 2:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > At least not for me. According to various things I found on the
> > Internet, it's now required that you codesign your binaries and give
> > them an entitlement in order to generate core dumps:
>
> Still works for me, at least on machines where I have SIP turned
> off.  Admittedly, Apple's busy making that a less and less desirable
> choice.

What exact version are you running?

--
Robert Haas
EDB: http://www.enterprisedb.com



Re: macOS Ventura won't generate core dumps

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Tue, Apr 9, 2024 at 2:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Still works for me, at least on machines where I have SIP turned
>> off.  Admittedly, Apple's busy making that a less and less desirable
>> choice.

> What exact version are you running?

Works for me on Sonoma 14.4.1 and Ventura 13.6.6, and has done
in many versions before those.

The usual gotchas apply: you need to have started the postmaster
under "ulimit -c unlimited", and the /cores directory has to be
writable by whatever user the postmaster is running as.  I have
occasionally seen system updates reduce the privileges on /cores,
although not recently.

            regards, tom lane



Re: macOS Ventura won't generate core dumps

From
Robert Haas
Date:
On Tue, Apr 9, 2024 at 3:44 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Works for me on Sonoma 14.4.1 and Ventura 13.6.6, and has done
> in many versions before those.
>
> The usual gotchas apply: you need to have started the postmaster
> under "ulimit -c unlimited", and the /cores directory has to be
> writable by whatever user the postmaster is running as.  I have
> occasionally seen system updates reduce the privileges on /cores,
> although not recently.

Interesting. I'm on Ventura 13.6.2. I think I've checked all of the
stuff you mention, but I'll do some more investigation.

--
Robert Haas
EDB: http://www.enterprisedb.com



Re: macOS Ventura won't generate core dumps

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Tue, Apr 9, 2024 at 3:44 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> The usual gotchas apply: you need to have started the postmaster
>> under "ulimit -c unlimited", and the /cores directory has to be
>> writable by whatever user the postmaster is running as.  I have
>> occasionally seen system updates reduce the privileges on /cores,
>> although not recently.

> Interesting. I'm on Ventura 13.6.2. I think I've checked all of the
> stuff you mention, but I'll do some more investigation.

Huh, that's odd.  One idea that comes to mind is that the core files
are frickin' large, for me typically around 3.5GB-4GB even for a
postmaster running with default shared memory sizes.  (I don't know
why, although it seems to me they were less ridiculous a few years
ago.)  Is it possible you're running out of space, or hitting a
quota of some sort?

            regards, tom lane



Re: macOS Ventura won't generate core dumps

From
Robert Haas
Date:
On Tue, Apr 9, 2024 at 4:34 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Huh, that's odd.  One idea that comes to mind is that the core files
> are frickin' large, for me typically around 3.5GB-4GB even for a
> postmaster running with default shared memory sizes.  (I don't know
> why, although it seems to me they were less ridiculous a few years
> ago.)  Is it possible you're running out of space, or hitting a
> quota of some sort?

Just to close the loop here, it turns out that this was caused by some
EDB-installed software, not macOS itself. The software in question
wasn't expected to cause this to happen, but did anyway.

We also discovered that core dumps weren't *completely* disabled. They
still occurred for segmentation faults, but not for abort traps.

So, if anyone's having a similar problem, check with your IT
department or try with a fresh OS install before blaming Apple. :-)

--
Robert Haas
EDB: http://www.enterprisedb.com