Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands. - Mailing list pgsql-hackers

On Wed, Jul 3, 2024, at 22:27, Joel Jacobson wrote:
> On Wed, Jul 3, 2024, at 20:57, Dean Rasheed wrote:
>> I wouldn't expect it to ever be off by more than 1, given that
>> MUL_GUARD_DIGITS = 2, which corresponds to 8 decimal digits, and the
>> number of digits in the smaller input (and hence the number of digit
>> products in each column) is limited to something like 16,000 NBASE
>> digits.
>
> OK, so then the cases I found where it was off by 2 for the mul_var_int() patch
> are unexpected?

Sorry, I meant off by 2 for the mul_var_small() patch, these cases that I found:

       var1        |      var2      | rscale_adjustment |   result   | numeric_mul_patch_small
-------------------+----------------+-------------------+------------+-------------------------
     8952.12658563 | 0.902315486665 |               -16 |  8077.6425 |               8077.6423
    0.881715409579 | 0.843165739371 |               -16 | 0.74343223 |              0.74343221
    0.905322758954 | 0.756905996850 |               -16 | 0.68524423 |              0.68524421
 8464.043170546608 | 0.518100129611 |               -20 |  4385.2219 |               4385.2217
 5253.006296984449 | 0.989308019355 |               -20 |  5196.8413 |               5196.8411
(5 rows)

Regards,
Joel