Request a Demo
Home   >   mailing lists

Re: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign - Mailing list pgsql-bugs

From Ayush Tiwari
Subject Re: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
Date
Msg-id CAJTYsWUurV6dBhnyd6R7W1w7Fy6FiG9-F_Swk_GDXQQpHoiciA@mail.gmail.com
Whole thread
In response to Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign  (王跃林 <violin0613@tju.edu.cn>)
Responses Re: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
List pgsql-bugs
Tree view
Hi,

On Sat, 13 Jun 2026 at 09:20, 王跃林 <violin0613@tju.edu.cn> wrote:
主题：Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
On Mon, May 25, 2026 at 10:58:05PM +0800, violin0613@tju.edu.cn wrote:
> PoC
> 
> File vuln_001.sql
> 
>  CREATE EXTENSION IF NOT EXISTS ltree;
>  â
>  WITH s AS (SELECT 'a'::ltree AS v),
>       l AS (SELECT (repeat('a.', 19999) || 'a')::ltree AS v)
>  SELECT (l.v > s.v) AS long_gt_short_expected_true,
>         (l.v < s.v) AS long_lt_short_expected_false
>  FROM s, l;
> 
> Process
> 
>  psql -h /tmp -p 36265 -U postgres -f vuln_001.sql
> 
> Results
> 
>   long_gt_short_expected_true | long_lt_short_expected_false
>  -----------------------------+------------------------------
>   f                           | t
> 
>    Both columns are inverted. long > short returned false, long < short
>    returned true.

This looks like a classic case of integer overflow that's
happening in ltree_compare function in ltree_op.c.

return (al->len - bl->len) * 10 * (an + 1);
return res * 10 * (an + 1);
return (a->numlevel - b->numlevel) * 10 * (an + 1);

I think the calculation should be done as int64, something of this sort:

    int64 v = (int64) (al->len - bl->len) * 10 * (an + 1);
    if (v > PG_INT32_MAX) return PG_INT32_MAX;
    if (v < PG_INT32_MIN) return PG_INT32_MIN;
    return (int) v;

And needed to adjust the ltree_penalty function too.

Attached is a draft patch for this, I guess we can add a helper
function too for the above conversion.

Thoughts?

Regards,
Ayush
Attachment

pgsql-bugs by date:

Previous
From: 王跃林
Date:
Subject: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
Next
From: PG Bug reporting form
Date:
Subject: BUG #19519: REPACK can fail due to missing chunk for toast value