pgsql: Improve the accuracy of numeric power() for integer exponents. - Mailing list pgsql-committers

From Dean Rasheed
Subject pgsql: Improve the accuracy of numeric power() for integer exponents.
Date
Msg-id E1olRcr-003D92-3k@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Improve the accuracy of numeric power() for integer exponents.

This makes the choice of result scale of numeric power() for integer
exponents consistent with the choice for non-integer exponents, and
with the result scale of other numeric functions. Specifically, the
result scale will be at least as large as the scale of either input,
and sufficient to ensure that the result has at least 16 significant
digits.

Formerly, the result scale was based only on the scale of the first
input, without taking into account the weight of the result. For
results with negative weight, that could lead to results with very few
or even no non-zero significant digits (e.g., 10.0 ^ (-18) produced
0.0000000000000000).

Fix this by moving responsibility for the choice of result scale into
power_var_int(), which already has code to estimate the result weight.

Per report by Adrian Klaver and suggested fix by Tom Lane.

No back-patch -- arguably this is a bug fix, but one which is easy to
work around, so it doesn't seem worth the risk of changing query
results in stable branches.

Discussion: https://postgr.es/m/12a40226-70ac-3a3b-3d3a-fdaf9e32d312%40aklaver.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/40c7fcbbed5d922e905f8032c5035826d0406980

Modified Files
--------------
src/backend/utils/adt/numeric.c           | 110 ++++++----
src/test/regress/expected/numeric.out     | 170 ++++++++++------
src/test/regress/expected/numeric_big.out | 324 +++++++++++++++---------------
src/test/regress/sql/numeric.sql          |   8 +-
src/test/regress/sql/numeric_big.sql      | 152 +++++++-------
5 files changed, 420 insertions(+), 344 deletions(-)


pgsql-committers by date:

Previous
From: Alvaro Herrera
Date:
Subject: pgsql: Use proper macro to access TransactionId
Next
From: Peter Eisentraut
Date:
Subject: pgsql: Make finding openssl program a configure or meson option