Re: Partitionwise join fails under GEQO - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Partitionwise join fails under GEQO
Date
Msg-id 1893673.1601999454@sss.pgh.pa.us
Whole thread Raw
In response to Re: Partitionwise join fails under GEQO  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
Responses Re: Partitionwise join fails under GEQO
List pgsql-hackers
Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> writes:
> On Mon, Oct 5, 2020 at 11:59 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> ... we could avoid the growth in eclass members for large partition sets
>> if we simply didn't store child eclass members, instead translating
>> on-the-fly when we need to deal with child rels.  I have a patch
>> about half done, but it won't be possible to determine the true
>> performance implications of that idea until it's all done.  More
>> later.

> If we translate more than ones, every time someone comes searching for
> an EC member, we will leak memory in the planner memory context, which
> anyway gets bloated because of the huge number of translations even
> when done only once. So that needs to be done rather carefully.

I'm not terribly concerned about that.  There's not a "huge number"
of such translations to be done --- it's more like one per index.
(Also, we could very possibly build the translations in a temp
context that gets reset every so often, if it becomes an issue.)

I am a bit worried about whether we'll be spending a lot more cycles
to do the added translations; though some of that should be bought
back by having fewer EC members to compare to.  In any event, testing
a working patch will be a lot more illuminating than speculation.

>> Either approach would mean that add_child_join_rel_equivalences
>> goes away entirely, or at least doesn't need to store new em_is_child
>> entries anymore, causing the memory bloat issue to become moot.
>> So maybe we should just fix the wrong-context issue for now, and
>> live with the GEQO bloat in the back branches.

> Yes, I agree with that. For now your patch fixing the wrong context
> issue is good enough.

Done for now.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Pavel Borisov
Date:
Subject: Re: Yet another fast GiST build
Next
From: Tomas Vondra
Date:
Subject: Re: [HACKERS] Custom compression methods