Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets - Mailing list pgsql-bugs

From Jeff Davis
Subject Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets
Date
Msg-id 1355185776.13316.4.camel@sussancws0025
Whole thread Raw
In response to Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets
Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets
List pgsql-bugs
On Sun, 2012-12-09 at 17:45 -0500, Tom Lane wrote:
> Yeah.  After looking at other uses of my_log2, it seems like
> hash_estimate_size and hash_select_dirsize probably should also
> bound their inputs to avoid overflow of 1 << my_log2() calculations.
>
> Alternatively, maybe we should hack my_log2 to do that bounding.
> As-is, it seems like trouble waiting to happen.  This won't protect
> init_htab, which needs the shift result to fit in int not long.
> But my_log2 is just plain broken for inputs larger than LONG_MAX/2,
> anyway.

It looks like all of the callers, except two, immediately shift the
result. So perhaps it would be better to make a new function (something
like "ceil_pow2") that returns the lowest power of two greater than or
equal to the input, and it can return a long (bounded to +LONG_MAX).
Then, the caller can bound the result further if needed, which should be
less error-prone, because the caller would see that it returns a long
(and compiler, but I don't seem to get a warning for implicit long->int
conversions).

Both of the callers that actually want a log2 already assume that the
input is a power of two, so we can redefine my_log2 to require it.

Regards,
    Jeff Davis

pgsql-bugs by date:

Previous
From: postgres@tbruce.com
Date:
Subject: BUG #7750: pid file conflict in RedHat
Next
From: Tom Lane
Date:
Subject: Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets