Re: Question about the TODO, numerics, and division - Mailing list pgsql-hackers

From Gregory Stark
Subject Re: Question about the TODO, numerics, and division
Date
Msg-id 87fy7yhhxk.fsf@stark.xeocode.com
Whole thread Raw
In response to Re: Question about the TODO, numerics, and division  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> but I wouldn't want to defend the details of the rule about how many
> fractional digits out given so many fractional digits in.

In particular it seems to me this is poor:

postgres=# select 1::numeric/10;       ?column?        
------------------------0.10000000000000000000
(1 row)


In an ideal world it seems to me that if you're dividing by a number with only
factors of 2 and 5 you should only gain as many digits as necessary to
represent the result exactly.

At the very least if you're dividing by a power of ten we should just move the
decimal place and keep the same overall precision.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: Martijn van Oosterhout
Date:
Subject: Re: Question about the TODO, numerics, and division
Next
From: Alvaro Herrera
Date:
Subject: Re: GSoC's possible project