Re: Inaccurate results from numeric ln(), log(), exp() and pow() - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Inaccurate results from numeric ln(), log(), exp() and pow()
Date
Msg-id 4514.1449777733@sss.pgh.pa.us
Whole thread Raw
In response to Re: Inaccurate results from numeric ln(), log(), exp() and pow()  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Inaccurate results from numeric ln(), log(), exp() and pow()
Re: Inaccurate results from numeric ln(), log(), exp() and pow()
List pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Dec 10, 2015 at 2:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> That's on me as author of the commit message, I guess.  The rscale
>> in most of these functions is exactly the number of fraction digits
>> that will be emitted, and we changed the rules for computing it.
>> Not by much, in most cases.  I don't think we should be too worried
>> about being bug-compatible with the old behavior.

> It seems to be a loss of 4 digits in every case I've seen.

I wouldn't have a problem with, say, throwing in an extra DEC_DIGITS worth
of rscale in each of these functions so that the discrepancies tend to
favor more significant digits out, rather than fewer.  I don't know that
it's worth trying to guarantee that the result is never fewer digits than
before, and I certainly wouldn't want to make the rules a lot more complex
than what's there now.  But perhaps we could cover most cases easily.

Dean, do you want to recheck the patch with an eye to that?
        regards, tom lane



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Inaccurate results from numeric ln(), log(), exp() and pow()
Next
From: Alvaro Herrera
Date:
Subject: Re: Re: In-core regression tests for replication, cascading, archiving, PITR, etc.