Re: Arbitrary precision modulo operation - Mailing list pgsql-general

From Dann Corbit
Subject Re: Arbitrary precision modulo operation
Date
Msg-id D90A5A6C612A39408103E6ECDD77B8299CAA49@voyager.corporate.connx.com
Whole thread Raw
In response to Arbitrary precision modulo operation  (Chadwick Boggs <chadwickboggs@yahoo.com>)
List pgsql-general
> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> Sent: Thursday, April 29, 2004 2:31 PM
> To: Dann Corbit
> Cc: Bruno Wolff III; Paul Tillotson; pgsql-general@postgresql.org
> Subject: Re: [GENERAL] Arbitrary precision modulo operation
>
>
> "Dann Corbit" <DCorbit@connx.com> writes:
> > From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> >> Fine.  How many is that, exactly?
>
> > Here is what I would suggest:
> > Using the outline I proposed before (starting with a floating point
> > divide of DBL_DIG digits of precision), keep doubling the precision
> > until the precision is 5 digits larger than either operand.
>  If the last
> > doubling makes the precision larger (quite likely) simply
> reduce it to
> > the smaller margin.
>
> And this guarantees a correct answer why?
>
> AFAIK div_var is already correct per its spec, which is that it
> generates an answer rounded to the requested number of digits.
> The question at hand is what number of digits to request.
>
> After thinking about it I don't see any reason that DBL_DIG has
> anything to do with a non-surprising answer ... much less
> "DBL_DIG + 5"
> which seems picked out of the air ...

DBL_DIG is there (from float.h) to define the precision of the first
operation.  After that, the precision doubles each time.

5 was pretty much picked out of the air (it would be good enough to
ensure correctness but is overkill).  I assumed that you could lose up
to one ULP per operation.

Do it any way you like.  It was just a suggestion.  It is clear that mod
is not working correctly now, and I was trying to be helpful.

pgsql-general by date:

Previous
From: Jean-Luc Lachance
Date:
Subject: Re: Timestamp problems...wrong weeks.
Next
From: Tom Lane
Date:
Subject: Re: Arbitrary precision modulo operation