Re: Incorrect results from numeric round() and trunc() - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: Incorrect results from numeric round() and trunc()
Date
Msg-id CAEZATCWhg+6pFhmmeCPw8yw4c7JqwYVs-54S4kt4Jpo7ZOmc_g@mail.gmail.com
Whole thread Raw
In response to Re: Incorrect results from numeric round() and trunc()  ("Joel Jacobson" <joel@compiler.org>)
Responses Re: Incorrect results from numeric round() and trunc()
List pgsql-hackers
On Mon, 8 Jul 2024 at 00:40, Joel Jacobson <joel@compiler.org> wrote:
>
> On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
> > I've also tidied up a bit by replacing all instances of SHRT_MAX with
> > a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
> > describes the limit, as used in various other overflow checks.
>
> Having thought a bit more on this, I think we probably need a
> DEC_DIGITS sensitive definition of NUMERIC_WEIGHT_MAX,
> since per spec the max range for numeric is 0x20000 (131072)
> decimal digits.
>

No, the maximum weight is determined by the use of int16 to store the
weight. Therefore if you did reduce DEC_DIGITS to 1 or 2, the number
of decimal digits allowed before the decimal point would be reduced
too.

Regards,
Dean



pgsql-hackers by date:

Previous
From: "Zhijie Hou (Fujitsu)"
Date:
Subject: RE: Conflict Detection and Resolution
Next
From: Matthias van de Meent
Date:
Subject: Re: Parallel CREATE INDEX for GIN indexes