Hi,
contrib/btree_gist's float4/float8 GiST opclasses compared keys with
raw C operators (==, <, >). Under IEEE 754 every comparison involving
NaN returns false, so GiST disagreed with the regular btree opclass,
which uses float[4|8]_cmp_internal() (all NaNs equal, NaN sorts after
every non-NaN value).
The disagreement was reachable from SQL:
* an index scan on a GiST index over a float column returned no rows
for `WHERE a = 'NaN'`, while a sequential scan returned the NaN row;
* `EXCLUDE USING gist (a WITH =)` accepted two rows whose key was NaN,
because gbt_float8eq(NaN, NaN) was false;
* an RLS predicate `USING (a != 'NaN')` leaked NaN rows on index scan.
The fix swaps the five gbt_float{4,8}{gt,ge,eq,le,lt} comparators and
the picksplit gbt_float{4,8}key_cmp for the NaN-aware
float{4,8}_{gt,ge,eq,le,lt} / float{4,8}_cmp_internal() helpers in
utils/float.h — same total order the btree opclass already uses. No
on-disk format change.
Added regression coverage in contrib/btree_gist/sql/float{4,8}.sql for
the three scenarios above. `make check` is green: core 245/245 +
contrib btree_gist 32/32 + all 48 contrib modules.
Originally reported as BUG #19501 and BUG #19524:
https://www.postgresql.org/message-id/19501-3bff3bbc97f1e7c9%40postgresql.org https://www.postgresql.org/message-id/19524-9559d302c8455664%40postgresql.org I used an LLM (Claude Code) to help with the analysis and to draft the
patch. I reviewed and tested the diff myself before sending.
Regards,
Bill Kim
