Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning - Mailing list pgsql-hackers

From David Rowley
Subject Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
Date
Msg-id CAApHDvoy=CwvN_jep3Eh3kJ61xfdiQcMFvk8B=q5kr8r1k2FzQ@mail.gmail.com
Whole thread Raw
In response to Reducing memory consumed by RestrictInfo list translations in partitionwise join planning  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
Responses Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
List pgsql-hackers
On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
> 0001 - to measure memory consumption during planning. This is the same
> one as attached to [1].

I see you're recording the difference in the CurrentMemoryContext of
palloc'd memory before and after planning.  That won't really alert us
to problems if the planner briefly allocates, say 12GBs of memory
during, say the join search then quickly pfree's it again.  unless
it's an oversized chunk, aset.c won't free() any memory until
MemoryContextReset(). Chunks just go onto a freelist for reuse later.
So at the end of planning, the context may still have that 12GBs
malloc'd, yet your new EXPLAIN ANALYZE property might end up just
reporting a tiny fraction of that.

I wonder if it would be more useful to just have ExplainOneQuery()
create a new memory context and switch to that and just report the
context's mem_allocated at the end.

It's also slightly annoying that these planner-related summary outputs
are linked to EXPLAIN ANALYZE. We could be showing them in EXPLAIN
without ANALYZE.  If we were to change that now, it might be a bit
annoying for the regression tests as we'd need to go and add SUMMARY
OFF in a load of places...

drowley@amd3990x:~/pg_src/src/test/regress/sql$ git grep -i "costs off" | wc -l
1592

hmm, that would cause a bit of churn... :-(

David



pgsql-hackers by date:

Previous
From: vignesh C
Date:
Subject: Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Next
From: Amit Kapila
Date:
Subject: Re: Handle infinite recursion in logical replication setup