On Tue, Aug 13, 2024, at 12:23, Dean Rasheed wrote:
> On Tue, 13 Aug 2024 at 08:49, Joel Jacobson <joel@compiler.org> wrote:
>>
>> I reran the tests and v5 produces much fewer diffs than v4.
>> Not sure if the remaining ones are problematic or not.
...
>
> Yes, that's exactly the sort of thing you'd expect to see. The exact
> product of var1 and var2 in that case is
>
> 0.0000_0000_0000_0003_2871_9239_2308_5000_4574_2504_736
>
> so numeric_mul_rscale() with the patch is producing the correctly
> rounded result, and "expected" is the result from HEAD, which is off
> by 1 in the final digit.
>
> To make it easier to hit such cases, I tested with the attached test
> script, which intentionally produces pairs of numbers whose product
> contains '5' followed by 5 zeros, and rounds at the digit before the
> '5', so the correct answer should round up, but the truncated product
> is quite likely not to do so.
>
> With HEAD, this gives 710,017 out of 1,000,000 cases that are off by 1
> in the final digit (always 1 too low in the final digit), and with the
> v5 patch, it gives 282,595 cases. Furthermore, it's an exact subset:
>
> select count(*) from diffs1; -- HEAD
> count
> --------
> 710017
> (1 row)
>
> pgdevel=# select count(*) from diffs2; -- v5 patch
> count
> --------
> 282595
> (1 row)
>
> select * from diffs2 except select * from diffs1;
> n | z | m | w | x | y | expected | numeric_mul_rscale
> ---+---+---+---+---+---+----------+--------------------
> (0 rows)
>
> which is exactly what I was hoping to see (no cases where the patch
> made it less accurate).
Nice. I got the same results:
select count(*) from diffs_head;
count
--------
710017
(1 row)
select count(*) from diffs_v4;
count
--------
344045
(1 row)
select count(*) from diffs_v5;
count
--------
282595
(1 row)
select count(*) from (select * from diffs_v4 except select * from diffs_head) as q;
count
-------
37236
(1 row)
select count(*) from (select * from diffs_v5 except select * from diffs_head) as q;
count
-------
0
(1 row)
I think this is acceptable, since it produces more correct results.
Regards,
Joel
