Re: Optimize mul_var() for var1ndigits >= 8 - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Optimize mul_var() for var1ndigits >= 8
Date
Msg-id CAEZATCWRT+c1nY_=hCDg0Lwn-QPe5kQSTXMRX+SdMSYewbf3ow@mail.gmail.com
Whole thread Raw
In response to Re: Optimize mul_var() for var1ndigits >= 8  ("Joel Jacobson" <joel@compiler.org>)
Responses Re: Optimize mul_var() for var1ndigits >= 8
List pgsql-hackers
On Wed, 14 Aug 2024 at 07:31, Joel Jacobson <joel@compiler.org> wrote:
>
> I think this is acceptable, since it produces more correct results.
>

Thanks for checking. I did a bit more testing myself and didn't see
any problems, so I have committed both these patches.

> In addition, I've traced the rscale_adjustment -15 mul_var() calls to originate
> from numeric_exp() and numeric_power(), so I thought it would be good to
> brute-force test those as well, to get an idea of the probability of different
> results from those functions.
>
> Brute-force testing of course doesn't prove it's impossible to happen,
> but millions of inputs didn't cause any observable differences in the
> returned results, so I think it's at least very improbable to
> happen in practice.
>

Indeed, there certainly will be cases where the result changes. I saw
some with ln(), for which HEAD rounded the final digit the wrong way,
and the result is now correct, but the opposite cannot be ruled out
either, since these functions are inherently inexact. The aim is to
have them generate the correctly rounded result in the vast majority
of cases, while accepting an occasional off-by-one error in the final
digit. Having them generate the correct result in all cases is
certainly possible, but would require a fair bit of additional code
that probably isn't worth the effort.

In my testing, exp() rounded the final digit incorrectly with a
probability of roughly 1 in 50-100 million when computing results with
a handful of digits (consistent with the "+8" digits added to
"sig_digits"), rising to roughly 1 in 5-10 million when computing
around 1000 digits (presumably because we don't take into account the
number of Taylor series terms when deciding on the local rscale). That
wasn't affected significantly by the patch, and it's not surprising
that you saw nothing with brute-force testing.

In any case, I'm pretty content with those results.

Regards,
Dean



pgsql-hackers by date:

Previous
From: Ilia Evdokimov
Date:
Subject: Re: Vacuum statistics
Next
From: Junwang Zhao
Date:
Subject: Re: [Patch] remove duplicated smgrclose