Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands. - Mailing list pgsql-hackers

From Joel Jacobson
Subject Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.
Date
Msg-id 0459ded5-6daf-4bf0-9ab0-e1f11ab15ec6@app.fastmail.com
Whole thread Raw
In response to Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.  ("Joel Jacobson" <joel@compiler.org>)
Responses Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands.
List pgsql-hackers
On Tue, Jul 2, 2024, at 11:05, Joel Jacobson wrote:
> On Tue, Jul 2, 2024, at 10:22, Dean Rasheed wrote:
>> Shortly after posting that, I realised that there was a small bug. This bit:
>>
>>             case 2:
>>                 newdig = (int) var1digits[1] * var2digits[res_ndigits - 4];
>>
>> isn't quite right in the case where rscale is less than the full
>> result. In that case, the least significant result digit has a
>> contribution from one more var2 digit, so it needs to be:
>>
>>                 newdig = (int) var1digits[1] * var2digits[res_ndigits - 4];
>>                 if (res_ndigits - 3 < var2ndigits)
>>                     newdig += (int) var1digits[0] * var2digits[res_ndigits - 3];
>>

Just to be able to verify mul_var() is working as expected when
rscale is less than the full result, I've added a numeric_mul_patched()
function that takes a third rscale_adjustment parameter:

I've tried to get a different result with and without the fix,
but I'm failing to hit the bug.

Can you think of an example that should trigger the bug?

Example usage:

SELECT numeric_mul_patched(9999.999,2,0);
var1ndigits=1 var2ndigits=2 rscale=3
 numeric_mul_patched
---------------------
           19999.998
(1 row)

SELECT numeric_mul_patched(9999.999,2,1);
var1ndigits=1 var2ndigits=2 rscale=4
 numeric_mul_patched
---------------------
          19999.9980
(1 row)

SELECT numeric_mul_patched(9999.999,2,-1);
var1ndigits=1 var2ndigits=2 rscale=2
 numeric_mul_patched
---------------------
            20000.00
(1 row)

SELECT numeric_mul_patched(9999.999,2,-2);
var1ndigits=1 var2ndigits=2 rscale=1
 numeric_mul_patched
---------------------
             20000.0
(1 row)

/Joel



pgsql-hackers by date:

Previous
From: Jim Jones
Date:
Subject: Re: New function normal_rand_array function to contrib/tablefunc.
Next
From: "Joel Jacobson"
Date:
Subject: Re: Add pg_get_acl() function get the ACL for a database object