# select 9223372036854775807 = 9223372036854775807::float;?column?
----------t
(1 row)
# select 9223372036854775807 = 9223372036854775807.0::float;?column?
----------t
(1 row)
# select 9223372036854775807 = 9223372036854775807.0000::numeric;?column?
----------t
(1 row)
This and the fact that it's still possible to find the row using the
original value would seem to indicate that the rounding is just a display
artifact..
----- Original Message -----
From: "Oliver Elphick" <olly@lfix.co.uk>
To: "Josh Berkus" <josh@agliodbs.com>
Cc: "Adam Lancaster" <alancaster@preventsys.com>; <pgsql-sql@postgresql.org>
Sent: Wednesday, August 25, 2004 5:21 AM
Subject: Re: [SQL] Possible rounding error of large float values?
> On Tue, 2004-08-24 at 20:52, Josh Berkus wrote:
> > Adam,
> >
> > > 9223372036854775807
> > >
> > > It gets selected out as:
> > >
> > > 9.22337203685478E18
> >
> > This is a property of FLOAT data types. They round.
> >
> > > Which appears to be rounded. When we cast it to numeric type we get:
> > >
> > > 9223372036854780000
> > >
> > > Which also is rounded. It is still possible to find the row using the
> > > original value
> >
> > Hmmm ... is 15 digits the limit of NUMERIC? It may be.
>
> It must be the limit of float. Numeric can't produce any more than it
> was given and it was cast from float.
>
> bray=# select 653596708775675750507850507570708696432 ::numeric;
> numeric
> -----------------------------------------
> 653596708775675750507850507570708696432
> (1 row)
>
> --
> Oliver Elphick olly@lfix.co.uk
> Isle of Wight http://www.lfix.co.uk/oliver
> GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA
> ========================================
> "I saw in the night visions, and, behold, one like the
> Son of man came with the clouds of heaven, and came to
> the Ancient of days, and they brought him near before
> him. And there was given him dominion, and glory, and
> a kingdom, that all people, nations, and languages,
> should serve him; his dominion is an everlasting
> dominion, which shall not pass away, and his kingdom
> that which shall not be destroyed."
> Daniel 7:13,14
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
