Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error - Mailing list pgsql-bugs

From Andrew Gierth
Subject Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Date
Msg-id 874lc7sb1a.fsf@news-spur.riddles.org.uk
Whole thread Raw
In response to BUG #15519: Casting float4 into int4 gets the wrong sign instead of"integer out of range" error  (PG Bug reporting form <noreply@postgresql.org>)
Responses Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
List pgsql-bugs
>>>>> "PG" == PG Bug reporting form <noreply@postgresql.org> writes:

 PG> The following bug has been logged on the website:
 PG> Bug reference:      15519
 PG> Logged by:          Victor Petrovykh
 PG> Email address:      victor@magic.io
 PG> PostgreSQL version: 10.5
 PG> Operating system:   Gentoo Linux 4.14.20
 PG> Description:        

 PG> Offending examples:
 PG> SELECT ((2147483647::float4) - 1.0::float4)::int4;
 PG> SELECT ((2147483590::float4) - 1.0::float4)::int4;
 PG> SELECT ((2147483647::float4) + 1.0::float4)::int4;

 PG> They all produce the same result: -2147483648

The bug here is that the ftoi4 conversion function thinks it can do
this:

    if (num < INT_MIN || num > INT_MAX || isnan(num))

but this causes INT_MIN and INT_MAX to be implicitly converted to floats
(not doubles), which can't represent their values exactly and ends up
with the conditions being false in the edge cases. (Specifically,
(float)INT_MAX is a bit larger than INT_MAX.)

Probably this should be changed to use ((float8) INT_MAX) etc. instead?

 PG> I understand that a float4 cannot represent large integers with the
 PG> same precision as int4, that's OK. What surprised me is that
 PG> instead of getting an "overflow error" or "integer out of range" I
 PG> simply got a negative result for a value that is actually close to
 PG> maximum int4. To contrast this, the query:

 PG> SELECT ((2147483647::float4) + 200.0::float4)::int4;
 PG> The above produces the expected "ERROR:  integer out of range"

Likewise you also get that error if you cast the float4 to a float8
before casting to integer.

-- 
Andrew (irc:RhodiumToad)


pgsql-bugs by date:

Previous
From: PG Bug reporting form
Date:
Subject: BUG #15519: Casting float4 into int4 gets the wrong sign instead of"integer out of range" error
Next
From: Tom Lane
Date:
Subject: Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error