Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal - Mailing list pgsql-committers

From Dean Rasheed
Subject Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal
Date
Msg-id CAEZATCXV6bdUUDckMVD=PtHXRUKTPZeHpy538xWY+wW=DZKYhg@mail.gmail.com
Whole thread Raw
In response to Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-committers
On Sat, 10 Jul 2021 at 16:01, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> I think this needs a bit more thought.  Before, a case like
>         select 1e-16000 * 1e-16000;
> produced
>         ERROR:  value overflows numeric format
> Now you get an exact zero (with a lot of trailing zeroes, but still
> it's just zero).  Doesn't that represent catastrophic loss of
> precision?

Hmm, "overflow" isn't a great result for that case either. Zero is the
closest we can get to the exact result with a fixed number of digits
after the decimal point.

> In general, I'm disturbed that we just threw away the previous
> promise that numeric multiplication results were exact.  That
> seems like a pretty fundamental property --- which is stated
> in so many words in the manual, btw --- and I'm not sure I want
> to give it up.

Perhaps we should amend the statement about numeric multiplication to
say that it's exact within the limits of the numeric type's supported
scale, which we also document in the manual as 16383.

That seems a lot better than throwing an overflow error for a result
that isn't very big, which limits what's possible with numeric
multiplication to much less than 16383 digits.

Regards,
Dean



pgsql-committers by date:

Previous
From: Tom Lane
Date:
Subject: Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal
Next
From: Tom Lane
Date:
Subject: pgsql: Fix busted test for ldap_initialize.