Division by zero error in to_char(num, '9.9EEEE') - Mailing list pgsql-hackers

From Dean Rasheed
Subject Division by zero error in to_char(num, '9.9EEEE')
Date
Msg-id CAEZATCWhojfH4whaqgUKBe8D5jNHB8ytzemL-PnRx+KCTyMXmg@mail.gmail.com
Whole thread Raw
Responses Re: Division by zero error in to_char(num, '9.9EEEE')  (Dean Rasheed <dean.a.rasheed@gmail.com>)
List pgsql-hackers
While testing the numeric_power() patch in [1], I found this problem
trying to use to_char() to format very small numbers:

SELECT to_char(1.2e-1001, '9.9EEEE'); -- OK
  to_char
------------
  1.2e-1001

SELECT to_char(1.2e-1002, '9.9EEEE'); -- fails
ERROR:  division by zero

It turns out that the problem is in get_str_from_var_sci() which
attempts to divide the input by 1e-1002 to get the significand.
However, it is using power_var_int() to compute 1e-1002, which has a
maximum rscale of NUMERIC_MAX_DISPLAY_SCALE (1000), so it returns 0,
which is the correct answer to that scale, and then
get_str_from_var_sci() attempts to divide by that.

Rather than messing with power_var_int(), I think the simplest
solution is to just introduce a new local function, as in the attached
patch. This directly constructs 10^n, for integer n, which is pretty
trivial, and doesn't need any numeric multiplication or rounding.

Regards,
Dean

[1] https://www.postgresql.org/message-id/CAEZATCUWUV_BP41Ob7QY12oF%2BqDxjTWfDpkdkcOOuojrDvOLxw%40mail.gmail.com

Attachment

pgsql-hackers by date:

Previous
From: Richard Guo
Date:
Subject: Re: Problem about postponing gathering partial paths for topmost scan/join rel
Next
From: Geoff Winkless
Date:
Subject: Re: Replace l337sp34k in comments.