Postgres Pro
Downloads
Home   >   mailing lists

Re: Possible rounding error of large float values? - Mailing list pgsql-sql

From Josh Berkus
Subject Re: Possible rounding error of large float values?
Date
Msg-id 200408241252.30182.josh@agliodbs.com
Whole thread Raw
In response to Possible rounding error of large float values?  ("Adam Lancaster" <alancaster@preventsys.com>)
Responses Re: Possible rounding error of large float values?
List pgsql-sql
Tree view 
Adam,

> 9223372036854775807
>
> It gets selected out as:
>
> 9.22337203685478E18

This is a property of FLOAT data types.  They round.

> Which appears to be rounded. When we cast it to numeric type we get:
>
> 9223372036854780000
>
> Which also is rounded. It is still possible to find the row using the
> original value

Hmmm ... is 15 digits the limit of NUMERIC?   It may be.

> Is this expected behavior?

Yes.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco

pgsql-sql by date:

Previous
From: "Adam Lancaster"
Date:
Subject: Possible rounding error of large float values?
Next
From: Oliver Elphick
Date:
Subject: Re: Possible rounding error of large float values?