Re: BUG #3387: mod on non-integer returns bad result - Mailing list pgsql-bugs

From Bruce Momjian
Subject Re: BUG #3387: mod on non-integer returns bad result
Date
Msg-id 200707170454.l6H4spR05607@momjian.us
Whole thread Raw
In response to Re: BUG #3387: mod on non-integer returns bad result  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-bugs
This has been saved for the 8.4 release:

    http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Tom Lane wrote:
> Gregory Stark <stark@enterprisedb.com> writes:
> > In the case reported div_var was getting 70/70 = 0.99999. Which is
> > really just wrong.
>
> Agreed, but I think your proposed patch is just a band-aid.  The real
> problem with div_var is that it generates inaccurate output digits at
> all --- it's assuming that computing a few guard digits before rounding
> will suffice to ensure that all the delivered digits are correct, but
> I think there will always be corner cases where it fails.
>
> I just blew the dust off my old copy of Knuth vol 2, and see that his
> algorithm for multi-precision division generates output digits that are
> correct to start with (or at least he never needs to revisit a digit
> after moving on to the next).  ISTM we should go over to an approach
> like that.  For the truncated-output case the result will be exact,
> and for the rounded-output case you generate exactly one guard digit
> to see if it's >= base/2.
>
>             regards, tom lane

--
  Bruce Momjian  <bruce@momjian.us>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

pgsql-bugs by date:

Previous
From: "ITAGAKI Takahiro"
Date:
Subject: BUG #3453: Error on COPY TO/FROM 'non-ascii-path'
Next
From: "Hiroshi Saito"
Date:
Subject: Re: BUG #3453: Error on COPY TO/FROM 'non-ascii-path'