Re: Question about the TODO, numerics, and division - Mailing list pgsql-hackers

From Martijn van Oosterhout
Subject Re: Question about the TODO, numerics, and division
Date
Msg-id 20070321122655.GA13787@svana.org
Whole thread Raw
In response to Re: Question about the TODO, numerics, and division  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Tue, Mar 20, 2007 at 08:27:46PM -0400, Tom Lane wrote:
> and of course if you multiply that by 6 you get
>
> regression=# SELECT (10::numeric(2,0) / 6::numeric(2,0)) * 6;
>       ?column?
> ---------------------
>  10.0000000000000002
> (1 row)

The only way to fix that is to shift from fixed-point arithmetic to
something that manipulates rationals, where 10/6 can be represented
exactly.

Arithmatic on rationals is not terribly complicated, the real problem
comes with the fact that as the number of operations increase, your
numerator and denominator are going to tend to infinity in an attempt
to represent your number accuratly. That's probably a solved problem
too, but still...

Have  a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

pgsql-hackers by date:

Previous
From: Gregory Stark
Date:
Subject: Re: Question: pg_class attributes and race conditions ?
Next
From: Gregory Stark
Date:
Subject: Re: Question about the TODO, numerics, and division