Re: BUG #5975: Incorrect result from mod function with cast - Mailing list pgsql-bugs

From Tom Lane
Subject Re: BUG #5975: Incorrect result from mod function with cast
Date
Msg-id 9075.1302717300@sss.pgh.pa.us
Whole thread Raw
In response to BUG #5975: Incorrect result from mod function with cast  ("John Dickson" <jdickson@tnsi.com>)
List pgsql-bugs
"John Dickson" <jdickson@tnsi.com> writes:
> The following SQL (executed on 8.3.11) shows a result one higher than the
> maximum permissible remainder when combined with a cast:

> select 1129590 % 66,
>   cast(1129590 as numeric(21, 0)) % 66;

>  ?column? | ?column?
> ----------+----------
>         0 |       66

This is basically a roundoff issue.  It's fixed in 8.4 and up, as a
result of this patch:

Author: Tom Lane <tgl@sss.pgh.pa.us>
Branch: master Release: REL8_4_BR [a0fad9762] 2008-04-04 18:45:36 +0000

    Re-implement division for numeric values using the traditional "schoolbook"
    algorithm.  This is a good deal slower than our old roundoff-error-prone
    code for long inputs, so we keep the old code for use in the transcendental
    functions, where everything is approximate anyway.  Also create a
    user-accessible function div(numeric, numeric) to provide access to the
    exact result of trunc(x/y) --- since the regular numeric / operator will
    round off its result, simply computing that expression in SQL doesn't
    reliably give the desired answer.  This fixes bug #3387 and various related
    corner cases, and improves the usefulness of PG for high-precision integer
    arithmetic.

We felt at the time that it was inappropriate to back-patch such a
behavioral change, and I doubt that decision is going to change now.
I'd suggest considering an update to 8.4 if this is a critical issue
for you.

            regards, tom lane

pgsql-bugs by date:

Previous
From: Tom Lane
Date:
Subject: Re: Make fails if env var U set
Next
From: Tom Lane
Date:
Subject: Re: BUG #5974: UNION construct type cast gives poor error message