Re: Solving hash table overrun problems - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Solving hash table overrun problems
Date
Msg-id 27706.1109950928@sss.pgh.pa.us
Whole thread Raw
In response to Re: Solving hash table overrun problems  (Bruno Wolff III <bruno@wolff.to>)
Responses Re: Solving hash table overrun problems
List pgsql-hackers
Bruno Wolff III <bruno@wolff.to> writes:
>   Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> * Estimate the number of batches N using the planner's estimate.
>> We will always choose N a power of 2.  A tuple's batch number is
>> ((H div K) mod N).

> If K is way low this could be very slow.

How so?  You're not concerned about the time to do the division itself
are you?

>> * Now begin scanning the outer join input.  Tuples of batch number
>> zero (according to the current calculation) can be matched to the
>> current hashtable contents.  Tuples of higher batch numbers are dropped
>> into holding files for the outer input, one per batch.

> For new keys at this step do you know their final disposition so that making
> new hash entries won't be necessary?

Well, we probably have a pretty fair idea of N at this point, so it'll
usually be right --- but we reserve the right to increase N again later
in case we have to do so because one of the later inner batches is much
bigger than the zero batch we are currently processing.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Bruno Wolff III
Date:
Subject: Re: Solving hash table overrun problems
Next
From: Bruno Wolff III
Date:
Subject: Re: Solving hash table overrun problems