GiST index build versus NaN coordinates - Mailing list pgsql-hackers

From Tom Lane
Subject GiST index build versus NaN coordinates
Date
Msg-id 28685.1468246504@sss.pgh.pa.us
Whole thread Raw
Responses Re: GiST index build versus NaN coordinates  (Emre Hasegeli <emre@hasegeli.com>)
Re: GiST index build versus NaN coordinates  (Andreas Seltenreich <seltenreich@gmx.de>)
Re: GiST index build versus NaN coordinates  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
I looked into the problem reported in bug #14238,
https://www.postgresql.org/message-id/20160708151747.1426.60150@wrigleys.postgresql.org
The submitter was kind enough to give me a copy of the problem data,
and it turns out that the issue is that a few of the boxes contain
NaN coordinates.  Armed with that knowledge, it's trivial to reproduce:

regression=# create table foo (f1 box);
CREATE TABLE
regression=# insert into foo values ('(3,4),(nan,5)');
INSERT 0 1
regression=# insert into foo select box(point(x,x+1),point(x,x+1)) from generate_series(1,1000) x;
INSERT 0 1000
regression=# create index on foo using gist(f1);
-- hangs, does not respond to control-C

The infinite loop is at lines 613ff in gistproc.c: once rightLower
contains a NaN, the test
           while (i1 < nentries && rightLower == intervalsLower[i1].lower)

can never succeed, so i1 never increments again and the loop cannot exit.

I think that probably the most reasonable answer is to replace all the
raw "double" comparisons in this code with float8_cmp_internal() or
something that's the moral equivalent of that.  Does anyone want to
propose something else?

More generally, this example makes me fearful that NaN coordinates in
geometric values are likely to cause all sorts of issues.  It's too late
to disallow them, probably, but I wonder how can we identify other bugs
of this ilk.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: pg_hba_lookup function to get all matching pg_hba.conf entries
Next
From: Stephen Frost
Date:
Subject: Re: remove checkpoint_warning