Re: Optimize planner memory consumption for huge arrays - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Optimize planner memory consumption for huge arrays
Date
Msg-id 1797494.1708960858@sss.pgh.pa.us
Whole thread Raw
In response to Re: Optimize planner memory consumption for huge arrays  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
List pgsql-hackers
Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
> On 2/25/24 17:29, Tom Lane wrote:
>> Yeah.  Also: once we had such an idea, it'd be very tempting to apply
>> it to other frequently-reset contexts like the executor's per-tuple
>> evaluation contexts.  I'm not quite prepared to argue that
>> MemoryContextReset should just act that way all the time ... but
>> it's sure interesting to think about.

> Do the context resets consume enough time to make this measurable?

I think they do.  We previously invented the "isReset" mechanism to
eliminate work in the case of exactly zero allocations since the
last reset, and that made a very measurable difference at the time,
even though you'd think the amount of work saved would be negligible.
This idea seems like it might be able to supersede that one and win
in a larger fraction of cases.

> +1 to disable this optimization in assert-enabled builds. I guess we'd
> invent a new constant to disable it, and tie it to USE_ASSERT_CHECKING
> (similar to CLOBBER_FREED_MEMORY, for example).

> Thinking about CLOBBER_FREED_MEMORY, could it be useful to still clobber
> the memory, even if we don't actually reset the context?

I think in any case where we were trying to support debugging, we'd
just disable the optimization, so that reset always resets.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: Running the fdw test from the terminal crashes into the core-dump
Next
From: Andy Fan
Date:
Subject: Re: Shared detoast Datum proposal