Re: Optimize numeric multiplication for one and two base-NBASE digit multiplicands. - Mailing list pgsql-hackers

On Tue, Jul 2, 2024, at 21:55, Joel Jacobson wrote:
> On Tue, Jul 2, 2024, at 20:53, Joel Jacobson wrote:
>> Trying to wrap my head around what could cause this.

I found the bug in the case 3 code,
and it turns out the same type of bug also exists in the case 2 code:

            case 2:
                newdig = (int) var1digits[1] * var2digits[res_ndigits - 4];

The problem here is that res_ndigits could become less than 4,
if rscale is low enough,
and then we would try to access a negative array index in var2digits.

Fix:

            case 2:
                if (res_ndigits - 4 >= 0 && res_ndigits - 4 < var2ndigits)
                    newdig = (int) var1digits[1] * var2digits[res_ndigits - 4];
                else
                    newdig = 0;

Another problem in the case 2 code:

                if (res_ndigits - 3 < var2ndigits)
                    newdig += (int) var1digits[0] * var2digits[res_ndigits - 3];

Fix:

                if (res_ndigits - 3 >= 0 && res_ndigits - 3 < var2ndigits)
                    newdig += (int) var1digits[0] * var2digits[res_ndigits - 3];

New version attached that fixes both the case 2 and case 3 code.

Regards,
Joel