Re: Solving hash table overrun problems - Mailing list pgsql-hackers

From Bruno Wolff III
Subject Re: Solving hash table overrun problems
Date
Msg-id 20050304154607.GA19919@wolff.to
Whole thread Raw
In response to Solving hash table overrun problems  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Solving hash table overrun problems  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Thu, Mar 03, 2005 at 17:05:40 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> * Estimate the number of batches N using the planner's estimate.
> We will always choose N a power of 2.  A tuple's batch number is
> ((H div K) mod N).

If K is way low this could be very slow. Is there a way to do something
similar changing the hash function to H div KN? If you went down this
road you would probably want to use distinct primes for each new N.

> * Now begin scanning the outer join input.  Tuples of batch number
> zero (according to the current calculation) can be matched to the
> current hashtable contents.  Tuples of higher batch numbers are dropped
> into holding files for the outer input, one per batch.

For new keys at this step do you know their final disposition so that making
new hash entries won't be necessary?


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: bitmap AM design
Next
From: Tom Lane
Date:
Subject: Re: Solving hash table overrun problems