Re: accounting for memory used for BufFile during hash joins - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: accounting for memory used for BufFile during hash joins
Date
Msg-id 20190507131742.nixczzyyyuuoig7m@development
Whole thread Raw
In response to Re: accounting for memory used for BufFile during hash joins  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: accounting for memory used for BufFile during hash joins
List pgsql-hackers
On Mon, May 06, 2019 at 11:18:28PM -0400, Tom Lane wrote:
>Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
>> Do we actually check how many duplicates are there during planning?
>
>Certainly that's part of the planner's cost estimates ... but it's
>only as good as the planner's statistical knowledge.
>

I'm looking at the code, and the only place where I see code dealing with
MCVs (probably the best place for info about duplicate values) is
estimate_hash_bucketsize in final_cost_hashjoin. That's not quite what I
had in mind - I was thinking more about something along the lines "See the
larget group of duplicate values, disable hash join if it can't fit into
work_mem at all."

Of course, if the input estimates are off, that may not work too well. It
would certainly not help the query failing with OOM, because that was a
case of severe underestimate.

Or did you mean some other piece of code that I have missed.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services




pgsql-hackers by date:

Previous
From: "Matsumura, Ryo"
Date:
Subject: RE: SQL statement PREPARE does not work in ECPG
Next
From: Tom Lane
Date:
Subject: Re: Unhappy about API changes in the no-fsm-for-small-rels patch