polygon && polygon and precision - Mailing list pgsql-general

From Patrick Dowler
Subject polygon && polygon and precision
Date
Msg-id CAFK8nrrGT53oz_jtX7EUn14G-KaY=HerrSWjTuoz4pn81znoyw@mail.gmail.com
Whole thread Raw
Responses Re: polygon && polygon and precision  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
List pgsql-general
I am using polygon datatype with some very small values and discovered that the overlaps operator appears to break down and give false positives for values smaller than ~1e-6. I have tested this on 9.5 (.3 and .16) and 10.10.

To reproduce (these examples only differ in the x-axis exponents) and should be false but the first one is t(rue)

select '((3.0e-07,-2),(9.0e-07,-2),(9.0e-07,1),(3.0e-07,1))'::polygon
    && '((2.0e-07,-0.1),(2.0e-07,0.1),(2.01e-07,0.1),(2.01e-07,-0.1))'::polygon;

select '((3.0e-06,-2),(9.0e-06,-2),(9.0e-06,1),(3.0e-06,1))'::polygon
    && '((2.0e-06,-0.1),(2.0e-06,0.1),(2.01e-06,0.1),(2.01e-06,-0.1))'::polygon;

Maybe suggests some single-precision floating point use in the calculations...

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada

pgsql-general by date:

Previous
From: Nicola Contu
Date:
Subject: Re: Streaming replication - 11.5
Next
From: Paul Förster
Date:
Subject: Patroni permanent replication slots