Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 - Mailing list pgsql-general

From Tom Lane
Subject Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2
Date
Msg-id 6426.965664708@sss.pgh.pa.us
Whole thread Raw
In response to Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2  (Philip Warner <pjw@rhyme.com.au>)
Responses Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2  (Philip Warner <pjw@rhyme.com.au>)
List pgsql-general
Philip Warner <pjw@rhyme.com.au> writes:
> pjw=# select float8(float4(10.1));
>       float8
> ------------------
>  10.1000003814697
> (1 row)

> I would have expected the latter to be at worst 10.10000000000000 +/-
> .00000000000001.

float4 is good to about 7 decimal digits (24 mantissa bits) on
IEEE-standard machines.  Thus the above result is actually closer
than you have any right to expect.

Don't they teach people about float arithmetic in CS 101 anymore?

            regards, tom lane

pgsql-general by date:

Previous
From: Thomas Lockhart
Date:
Subject: Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2
Next
From: Philip Warner
Date:
Subject: Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2