Re: numeric.c: Should MUL_GUARD_DIGITS be increased from 2 to 3? - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: numeric.c: Should MUL_GUARD_DIGITS be increased from 2 to 3?
Date
Msg-id CAEZATCUoUmcFELiDcbQ2WUmWaEr_Tce+yDMzC6gHw=u8C_tqdw@mail.gmail.com
Whole thread Raw
In response to numeric.c: Should MUL_GUARD_DIGITS be increased from 2 to 3?  ("Joel Jacobson" <joel@compiler.org>)
List pgsql-hackers
On Wed, 3 Jul 2024 at 08:36, Joel Jacobson <joel@compiler.org> wrote:
>
> Hello hackers,
>
> I have discovered a peculiar behavior in mul_var() when it is called with
> rscale=0, but the input variables have many decimal digits, resulting in a
> product with a .5 decimal part. Given that no decimals are requested by the
> caller, I would expect the result to be rounded up. However, it is instead
> truncated or rounded down.
>
> To investigate this further, I added an SQL-callable function,
> numeric_mul_patched(), which takes rscale_adjustment as a third input parameter.
> This allows calling mul_var() with varying rscale values.
>
> Here is an example demonstrating the issue:
>
> SELECT 5.51574061794 * 0.99715659165;
> 5.5000571150105152442010 -- exact result
>

No, as I mentioned on the other thread, that's not a bug, but perhaps
it's worth mentioning in the function's comment that it doesn't
guarantee correctly rounded results when rscale < var1->dscale +
var2->dscale. What it does do is "good enough" for the transcendental
functions that use it in this way, which are themselves inexact.

Note that increasing MUL_GUARD_DIGITS won't guarantee correctly
rounded results, no matter how much you increase it by. The only way
to guarantee that the result is correctly rounded in all cases is to
compute the full result and then round, which would be a lot slower.

Regards,
Dean



pgsql-hackers by date:

Previous
From: shveta malik
Date:
Subject: Re: Conflict Detection and Resolution
Next
From: Ranier Vilela
Date:
Subject: Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.