Re: intarray: fix an edge case int32 overflow bug - Mailing list pgsql-hackers

From David Rowley
Subject Re: intarray: fix an edge case int32 overflow bug
Date
Msg-id CAApHDvp2f_1ecHx+R_s9QtJAHO5AWwOCop3cW5q=0vox+-E+=A@mail.gmail.com
Whole thread Raw
In response to intarray: fix an edge case int32 overflow bug  (Chao Li <li.evan.chao@gmail.com>)
Responses Re: intarray: fix an edge case int32 overflow bug
Re: intarray: fix an edge case int32 overflow bug
List pgsql-hackers
On Sun, 4 Jan 2026 at 16:20, Chao Li <li.evan.chao@gmail.com> wrote:
> I noticed an int32 overflow problem in intarray’s compare_val_int4():
> ```
> /*
>  * Comparison function for binary search in mcelem array.
>  */
> static int
> compare_val_int4(const void *a, const void *b)
> {
>     int32       key = *(int32 *) a;
>     const Datum *t = (const Datum *) b;
>
>     return key - DatumGetInt32(*t);
> }
> ```
>
> As this function is a bsearch comparator, it is supposed to return >0, =0 or <0. However this function uses
subtractionwith two int32 and returns an int, which may result in an overflow. Say, key is INT32_MAX and *t is -1, the
returnvalue will be negative due to overflow. 

Nice find. Was that found by a static analyser or by eye?

I can take care of the overflow issue. I feel the test is a step too
far as it seems unlikely ever to be rebroken, but thanks for the
SQL-based test case to demonstrate the issue.

David



pgsql-hackers by date:

Previous
From: Pavel Stehule
Date:
Subject: Re: [PATCH] psql: add size-based sorting options (O/o) for tables and indexes
Next
From: David Rowley
Date:
Subject: Re: intarray: fix an edge case int32 overflow bug