Re: Optimize mul_var() for var1ndigits >= 8 - Mailing list pgsql-hackers

On Tue, Aug 13, 2024, at 00:56, Dean Rasheed wrote:
> On Mon, 12 Aug 2024 at 16:17, Joel Jacobson <joel@compiler.org> wrote:
>>
>> On Mon, Aug 12, 2024, at 17:14, Joel Jacobson wrote:
>> > The case found with the smallest rscale adjustment was this one:
>> > -[ RECORD 1 ]------+--------------------------------
>> > var1               | 0.0000000000009873307197037692
>> > var2               | 0.426697279270850
>> > rscale_adjustment  | -15
>> > expected           | 0.0000000000004212913318381285
>> > numeric_mul_rscale | 0.0000000000004212913318381284
>> > diff               | -0.0000000000000000000000000001
>>
>
> Hmm, interesting example. There will of course always be cases where
> the result isn't exact, but HEAD does produce the expected result in
> this case, and the intention is to always produce a result at least as
> accurate as HEAD, so it isn't working as expected.
>
..
> Updated patch attached, which fixes the above example and all the
> other differences produced by your test. I think, with a little
> thought, it ought to be possible to produce examples that round
> incorrectly in a more systematic (less brute-force) way. It should
> then be possible to construct examples where the patch differs from
> HEAD, but hopefully only by being more accurate, not less.

I reran the tests and v5 produces much fewer diffs than v4.
Not sure if the remaining ones are problematic or not.

joel@Joels-MBP postgresql % ./test-mul_var-init.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 1413
DROP TABLE
CREATE TABLE
 setseed
---------

(1 row)

INSERT 0 14130000
COPY 14130000

joel@Joels-MBP postgresql % ./test-mul_var-verify-v4.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 14130000
Expanded display is on.
-[ RECORD 1 ]------+--------------------------------
var1               | 0.0000000000009873307197037692
var2               | 0.426697279270850
rscale_adjustment  | -15
expected           | 0.0000000000004212913318381285
numeric_mul_rscale | 0.0000000000004212913318381284
diff               | -0.0000000000000000000000000001

Expanded display is off.
     diff     |  count
--------------+----------
  0.000e+00   | 14114384
  1.000e-108  |        1
  1.000e-211  |        1
  1.000e-220  |        2
  1.000e-228  |        6
  1.000e-232  |        2
  1.000e-235  |        1
  1.000e-28   |       13
  1.000e-36   |        1
  1.000e-51   |        2
  1.000e-67   |        1
  1.000e-68   |        1
  1.000e-80   |        1
 -1.000e-1024 |     2485
 -1.000e-108  |        3
 -1.000e-144  |     2520
 -1.000e-16   |     2514
 -1.000e-228  |        4
 -1.000e-232  |        1
 -1.000e-27   |       36
 -1.000e-28   |      538
 -1.000e-32   |     2513
 -1.000e-48   |     2473
 -1.000e-68   |        1
 -1.000e-80   |     2494
 -2.000e-16   |        2
(26 rows)

 rscale_adjustment | count
-------------------+-------
              -237 |     2
              -235 |     1
              -232 |     3
              -229 |     2
              -228 |     8
              -218 |     1
              -108 |     4
               -77 |     1
               -67 |     1
               -51 |     2
               -38 |     3
               -36 |     1
               -28 |     5
               -22 |    42
               -17 |     7
               -16 | 14959
               -15 |   574
(17 rows)

joel@Joels-MBP postgresql % ./test-mul_var-verify-v5.sh
HEAD is now at a67a49648d Rename C23 keyword
SET
DROP TABLE
CREATE TABLE
COPY 14130000
Expanded display is on.
-[ RECORD 1 ]------+-------------------------------
var1               | 0.0000000000000000489673392928
var2               | 6.713030439846337
rscale_adjustment  | -15
expected           | 0.0000000000000003287192392308
numeric_mul_rscale | 0.0000000000000003287192392309
diff               | 0.0000000000000000000000000001

Expanded display is off.
     diff     |  count
--------------+----------
  0.000e+00   | 14129971
  1.000e-1024 |        1
  1.000e-144  |        1
  1.000e-16   |        1
  1.000e-211  |        1
  1.000e-220  |        2
  1.000e-228  |        5
  1.000e-232  |        1
  1.000e-235  |        1
  1.000e-28   |        8
  1.000e-32   |        2
  1.000e-36   |        1
  1.000e-51   |        2
  1.000e-67   |        1
  1.000e-68   |        1
  1.000e-80   |        1
(16 rows)

 rscale_adjustment | count
-------------------+-------
              -237 |     1
              -235 |     1
              -232 |     1
              -229 |     2
              -228 |     4
              -218 |     1
               -77 |     1
               -67 |     1
               -51 |     2
               -38 |     1
               -36 |     1
               -28 |     2
               -17 |     4
               -16 |     5
               -15 |     2
(15 rows)

Regards,
Joel