BUG #18129: GiST index produces incorrect query results - Mailing list pgsql-bugs

The following bug has been logged on the website:

Bug reference:      18129
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 16.0
Operating system:   Ubuntu 22.04
Description:

The following test data manipulation:
sed '/{73,23,20}/!s/\([0-9]\+\)/\1,10\1,20\1,30\1/g' -i
contrib/intarray/data/test__int.data
(effectively multiplying contents of all arrays (except for one) in the
test by 3)

makes `make check contrib/intarray` fail, with random results, e.g.:
--- .../contrib/intarray/expected/_int.out        2023-09-21
21:44:46.967470822 +0300
+++ .../contrib/intarray/results/_int.out 2023-09-22 08:12:18.269739402
+0300
@@ -497,13 +497,13 @@
 SELECT count(*) from test__int WHERE a && '{23,50}';
  count 
 -------
-   403
+   395
 (1 row)
 
 SELECT count(*) from test__int WHERE a @@ '23|50';
  count 
 -------
-   403
+   395
 (1 row)
 
 SELECT count(*) from test__int WHERE a @> '{23,50}';
@@ -557,13 +557,13 @@
 SELECT count(*) from test__int WHERE a @@ '20 | !21';
  count 
 -------
-  6566
+  6478
 (1 row)
 
 SELECT count(*) from test__int WHERE a @@ '!20 & !21';
  count 
 -------
-  6343
+  6258
 (1 row)
 
 INSERT INTO test__int SELECT array(SELECT x FROM generate_series(1, 1001)
x); -- should fail
@@ -639,13 +639,13 @@
 SELECT count(*) from test__int WHERE a @@ '20 | !21';
  count 
 -------
-  6566
+  6381
 (1 row)
 
 SELECT count(*) from test__int WHERE a @@ '!20 & !21';
  count 
 -------
-  6343
+  6165
 (1 row)
 
 DROP INDEX text_idx;

(Here we deal with intarray's gist__int_ops, but I suspect that other gist
opclasses might be affected as well.)

First bad commit is 9155580fd, on which three different outcomes are
possible: test passes/test fails/assertion fails.
Assertion failures, that we had observed after 9155580fd, were fixed by
a7ee7c851 and 741b88435, but apparently there is another defect induced
by that gist implementation change.