> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> Sent: Thursday, April 29, 2004 2:31 PM
> To: Dann Corbit
> Cc: Bruno Wolff III; Paul Tillotson; pgsql-general@postgresql.org
> Subject: Re: [GENERAL] Arbitrary precision modulo operation
>
>
> "Dann Corbit" <DCorbit@connx.com> writes:
> > From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> >> Fine. How many is that, exactly?
>
> > Here is what I would suggest:
> > Using the outline I proposed before (starting with a floating point
> > divide of DBL_DIG digits of precision), keep doubling the precision
> > until the precision is 5 digits larger than either operand.
> If the last
> > doubling makes the precision larger (quite likely) simply
> reduce it to
> > the smaller margin.
>
> And this guarantees a correct answer why?
>
> AFAIK div_var is already correct per its spec, which is that it
> generates an answer rounded to the requested number of digits.
> The question at hand is what number of digits to request.
>
> After thinking about it I don't see any reason that DBL_DIG has
> anything to do with a non-surprising answer ... much less
> "DBL_DIG + 5"
> which seems picked out of the air ...
DBL_DIG is there (from float.h) to define the precision of the first
operation. After that, the precision doubles each time.
5 was pretty much picked out of the air (it would be good enough to
ensure correctness but is overkill). I assumed that you could lose up
to one ULP per operation.
Do it any way you like. It was just a suggestion. It is clear that mod
is not working correctly now, and I was trying to be helpful.
