Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
Date
Msg-id da7a213a-cdf3-4f56-84b8-e30269e07282@enterprisedb.com
Whole thread Raw
In response to Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
Responses Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning
List pgsql-hackers
Hi,

After taking a look at the patch optimizing SpecialJoinInfo allocations,
I decided to take a quick look at this one too. I don't have any
specific comments on the code yet, but it seems quite a bit more complex
than the other patch ... it's introducing a HTAB into the optimizer,
surely that has costs too.

I started by doing the same test as with the other patch, comparing
master to the two patches (applied independently and both). And I got
about this (in MB):

  tables    master     sjinfo    rinfo      both
  -----------------------------------------------
       2        37         36       34        33
       3       138        129      122       113
       4       421        376      363       318
       5      1495       1254     1172       931

Unlike the SpecialJoinInfo patch, I haven't seen any reduction in
planning time for this one.

The reduction in allocated memory is nice. I wonder what's allocating
the remaining memory, and we'd have to do to reduce it further.

However, this is a somewhat extreme example - it's joining 5 tables,
each with 1000 partitions, using a partition-wise join. It reduces the
amount of memory, but the planning time is still quite high (and
essentially the same as without the patch). So it's not like it'd make
them significantly more practical ... do we have any other ideas/plans
how to improve that?

AFAIK we don't expect this to improve "regular" cases with modest number
of tables / partitions, etc. But could it make them slower in some case?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Add system identifier to backup manifest
Next
From: Amit Langote
Date:
Subject: Re: Running the fdw test from the terminal crashes into the core-dump