Re: BUG #15519: Casting float4 into int4 gets the wrong sign insteadof "integer out of range" error - Mailing list pgsql-bugs

From Victor Petrovykh
Subject Re: BUG #15519: Casting float4 into int4 gets the wrong sign insteadof "integer out of range" error
Date
Msg-id CAJ-A5aNZjv+BSg08ZtMwsmiqucKcBeSirfYs16AnY5azD=ZHZw@mail.gmail.com
Whole thread Raw
In response to Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error  (Andrew Gierth <andrew@tao11.riddles.org.uk>)
List pgsql-bugs
I'm sorry I'm kinda new here.

Am I missing something in thinking that the cast of a float to int should produce an error if the sign of the original value doesn't match the sign of the cast result? Presumably you have access to both values at some point and checking sign bits for sameness should be simple. 

Also, I would say that doing some float4 arithmetic and then casting into int4 or int8 should be consistent, so if

SELECT (54610::float4 * 39324::float4)::int8;

gets me 2147483648, then 

SELECT (54610::float4 * 39324::float4)::int4;

should be an error rather than the unintuitive -2147483648.

The scenario that I'm pointing at is basically first doing some float4 arithmetic, casting the result into an int4 and expecting that the result (if it was without error) is actually about as close to the original float value as float4 precision would normally allow.

On Fri, Nov 23, 2018 at 7:45 PM Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:
>>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

 Tom> if (unlikely(num < (float4) INT_MIN || num >= (float4) INT_MAX || isnan(num)))

 >> if (num < (float4)INT_MIN || num >= -(float4)INT_MIN || ...

 Tom> Meh. Seems to me that's relying on pretty much the same
 Tom> assumptions

No, because we know that INT_MIN is always exactly representable as a
float (whereas INT_MAX is not), and therefore the cast result will not
depend on any rounding choices whether at compile or run time. Nor does
it depend on knowing that float4 can't represent INT_MAX exactly.

--
Andrew (irc:RhodiumToad)

pgsql-bugs by date:

Previous
From: Andrew Gierth
Date:
Subject: Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Next
From: Tom Lane
Date:
Subject: Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error