Re: BUG #3387: mod on non-integer returns bad result - Mailing list pgsql-bugs

From Gregory Stark
Subject Re: BUG #3387: mod on non-integer returns bad result
Date
Msg-id 87r6odb1k9.fsf@oxford.xeocode.com
Whole thread Raw
In response to Re: BUG #3387: mod on non-integer returns bad result  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: BUG #3387: mod on non-integer returns bad result  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-bugs
"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> Gregory Stark <stark@enterprisedb.com> writes:
>> The source of the problem is the floating point arithmetic which is used to do
>> the individual steps in the long division.
>
> I don't think so.  The ultimate source of the problem is that div_var
> can only report a finite number of digits.

In the case reported div_var was getting 70/70 = 0.99999. Which is really just
wrong. The only reason was because 1.0/70 isn't representable so (1.0/70) * 70
is slightly more than 1 which ... div_var really ought not have any trouble
representing an integer even in its finite number of digits.

--
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

pgsql-bugs by date:

Previous
From: Tom Lane
Date:
Subject: Re: BUG #3387: mod on non-integer returns bad result
Next
From: Tom Lane
Date:
Subject: Re: BUG #3387: mod on non-integer returns bad result