Re: Fixing geometic calculation - Mailing list pgsql-hackers

From Greg Stark
Subject Re: Fixing geometic calculation
Date
Msg-id 407d949e0908071148l1e225622g132bf02987a80993@mail.gmail.com
Whole thread Raw
In response to Re: Fixing geometic calculation  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Fixing geometic calculation  (Sam Mason <sam@samason.me.uk>)
List pgsql-hackers
On Fri, Aug 7, 2009 at 7:13 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> Sam Mason <sam@samason.me.uk> writes:
>> On Fri, Aug 07, 2009 at 12:50:39PM -0400, Tom Lane wrote:
>>> No, I'm worried about code that supposes that it can divide by (x - y)
>>> after testing that FPeq(x,y) is not true.  point_sl() for instance.
>
>> OK, but I'm still not sure what you're getting at.
>
> Underflow.  x!=y does not imply (x-y) != 0, if x and y are sufficiently
> small and close together.  The difference could underflow to zero.


Actually I don't think subtraction can underflow with IEEE floats but
I don't think we want to count on IEEE floats everywhere. Even if we
did there's the risk on intel that FPeq() gets called on values which
have just been calculated and are still in registers but then get
spilled to RAM and lose precision before the division happens.

--
greg
http://mit.edu/~gsstark/resume.pdf


pgsql-hackers by date:

Previous
From: Sam Mason
Date:
Subject: Re: Fixing geometic calculation
Next
From: Tom Lane
Date:
Subject: Re: "PANIC: cannot make new WAL entries during recovery" in the wild