Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning - Mailing list pgsql-hackers

From Ashutosh Bapat
Subject Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
Date
Msg-id CAExHW5vjRj_qRSn0wGbjs3S8P7cETBD9raoZkhPoV=q7n8u_Mw@mail.gmail.com
Whole thread Raw
In response to Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
Responses Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
List pgsql-hackers
Hi,

On Wed, Mar 26, 2025 at 4:59 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
>
> The averages are now more stable than the previous exercise. Some regressions seen with the first exercise are not
seenwith the other and some improvements seens with the first exercise are not seen with the second and vice versa. The
regressionspresent in both the exercises will not be seen, if I repeat the exercise a few more times. So I think those
regressionsor improvements seen with lower number of joins and lower number of partitions aren't real and they are
mostlywithin noise level. However, the improvements seen with higher numbers of joins and partitions are always there
irrespectiveof the number of times I repeat the exercise. Please note that we have only tried partitions upto 256.
Previousmeasurements have seen stable improvements in case of higher number of partitions. 
>

Further, I experimented with hash table size. Attached images have
four graphs for planning time and planner's memory consumption
measured for a 3-way join for initial has table sizes of 64, 128 and
256 respectively.
1. Planning time vs number of partitions with PWJ enabled
2. planning time vs number of partitions with PWJ disabled
3. Memory consumed vs number of partitions with PWJ enabled
4. Memory consumed vs number of partitions with PWJ disabled

Also find attached the spreadsheet containing the measurements and
also the charts.

In the graphs, one can observe that the lines corresponding to all
three hash table sizes are inseparable. That leads to a conclusion
that the planning time or memory consumption doesn't change with hash
table size.

As a result I am keeping the initial hash table size = 256L, same as
the previously attached patches.

--
Best Wishes,
Ashutosh Bapat

Attachment

pgsql-hackers by date:

Previous
From: Andrea Gelmini
Date:
Subject: Re: FileFallocate misbehaving on XFS
Next
From: Andrew Dunstan
Date:
Subject: Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote