Re: Make memory checking / sanitizing infrastructure better - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Make memory checking / sanitizing infrastructure better
Date
Msg-id dnp5vgairsqi5tsq5humtsyk55dzxsrzmoavqz47biryk6g6q4@2xvfyigx32ju
Whole thread
In response to Re: Make memory checking / sanitizing infrastructure better  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
List pgsql-hackers
Hi,

On 2026-10-08 14:56:24 +0530, Ashutosh Bapat wrote:
> On Thu, May 28, 2026 at 9:37 PM Andres Freund <andres@anarazel.de> wrote:
> > 3) I think we should make the memory context failures crash.  Perhaps by
> >    emitting WARNINGs and then crashing if any of them occurred.
> >
> >    As we trigger memory context checking during commit/abort handling, just
> >    using PANIC won't always reliably work, E.g. the AllocSetCheck() in
> >    AllocSetReset(), can trigger recursive PANICs due to the
> >    MemoryContextReset() in errstart(), if the corruption in in ErrorContext.
> >    Whether that's a real issue worth worrying about, IDK.
> >
> > 4) For this I prototyped making the valgrind annotations more generic and
> >    using the address sanitizer interface to mark memory as poisoned /
> >    unpoisoned.  That doesn't provide quite all the checking that valgrind can
> >    do (it doesn't track uninitialized memory), but it's considerably better
> >    than our memory context checking, and *much* *much* faster than valgrind.
> 
> IIUC, you are proposing to use sanitizer interface to implement
> crashing behaviour mentioned in 3? Or these two are separate things?

Separate. Using asan is pretty expensive (~2-3x slower tests, considerably
higher memory usage), there's absolutely no reason to not have our own memory
context checking infrastructure only really be usable if you use sanitizers.
I think there's also corruption that only our internal checking will find.

Running all tests without asan:

real    1m34.240s
user    6m27.832s
sys    4m20.235s

with asan:
real    3m31.336s
user    12m44.840s
sys    9m13.033s



> > 5) I think we should add a mode to all our allocators that just turns every
> >    allocation into a large allocation (perhaps with something slightly
> >    different for slab). In that mode asan, valgrind, et al would be more
> >    precise.  Of course that'll have rather deleterious performance effects,
> >    but I think it'd still be quite useful for debugging.
> 
> I think this has an advantage of its own, we will benefit from memory
> sanity check improvements in system calls without necessarily
> implementing them in PostgreSQL and also use-after-free failure not
> necessarily covered by other proposals.

I'm not really following, which memory sanity checking in system calls are you
thinking of?


FWIW, I'd started hacking on the asan piece a while back, here's my last WIP
patches on this topic. I ran out of time to work on this at the time, and
haven't found my way back since.  No guarantees that anything therein is
correct.

Greetings,

Andres Freund

Attachment

pgsql-hackers by date:

Previous
From: Zsolt Parragi
Date:
Subject: Re: REPACK: warn about skipping foreign partitions
Next
From: Rui Zhao
Date:
Subject: Re: fix more casting away of qualifiers