Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands. - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.
Date
Msg-id CAEZATCUozj6+46cpNU4ajB3iwUQMRJtTQOBui8gFgWLhEbb_rQ@mail.gmail.com
Whole thread Raw
In response to Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.  ("Joel Jacobson" <joel@compiler.org>)
List pgsql-hackers
On Tue, 2 Jul 2024 at 20:55, Joel Jacobson <joel@compiler.org> wrote:
>
> Interesting, I actually think there is a bug in the normal mul_var() code.
> Found a case that rounds down, when it should round up:
>
> Calling mul_var() with:
> var1=51.2945442386599
> var2=0.828548712212
> rscale=0
>
> returns 42, but I think it should return 43,
> since 51.2945442386599*0.828548712212=42.5000285724431241296446988
>
> But maybe this is expected and OK, having to do with MUL_GUARD_DIGITS?
>

No, that's not a bug. It's to be expected. The point of using only
MUL_GUARD_DIGITS extra digits is to get the correctly rounded result
*most of the time*, while saving a lot of effort by not computing the
full result.

The callers of mul_var() that ask for reduced rscale results are the
transcendental functions like ln_var() and exp_var(), which are
working to some limited precision intended to compute the final result
reasonably accurately to a particular rscale.

Regards,
Dean



pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: Use generation memory context for tuplestore.c
Next
From: Dilip Kumar
Date:
Subject: Re: Conflict Detection and Resolution