Re: Avoiding hash join batch explosions with extreme skew and weird stats - Mailing list pgsql-hackers

From Melanie Plageman
Subject Re: Avoiding hash join batch explosions with extreme skew and weird stats
Date
Msg-id CAAKRu_am1yFRKWKq5_zPQHDnfaDHQPM8xmqVmvta2HSHqjpD3w@mail.gmail.com
Whole thread Raw
In response to Re: Avoiding hash join batch explosions with extreme skew and weird stats  (Melanie Plageman <melanieplageman@gmail.com>)
Responses Re: Avoiding hash join batch explosions with extreme skew and weirdstats  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Re: Avoiding hash join batch explosions with extreme skew and weird stats  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers

On Tue, Apr 28, 2020 at 11:50 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 29/04/2020 05:03, Melanie Plageman wrote:
> I've attached a patch which should address some of the previous feedback
> about code complexity. Two of my co-workers and I wrote what is
> essentially a new prototype of the idea. It uses the main state machine
> to route emitting unmatched tuples instead of introducing a separate
> state. The logic for falling back is also more developed.

I haven't looked at the patch in detail, but thanks for the commit
message; it describes very well what this is all about. It would be nice
to copy that explanation to the top comment in nodeHashJoin.c in some
form. I think we're missing a high level explanation of how the batching
works even before this new patch, and that commit message does a good
job at it.


Thanks for taking a look, Heikki!

I made a few edits to the message and threw it into a draft patch (on
top of master, of course). I didn't want to junk up peoples' inboxes, so
I didn't start a separate thread, but, it will be pretty hard to
collaboratively edit the comment/ever register it for a commitfest if it
is wedged into this thread. What do you think?

--
Melanie Plageman
Attachment

pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Back-patch is necessary? Re: Don't try fetching future segment of aTLI.
Next
From: Fujii Masao
Date:
Subject: Re: +(pg_lsn, int8) and -(pg_lsn, int8) operators