Re: Small patch to improve safety of utf8_to_unicode(). - Mailing list pgsql-hackers

From Ayush Tiwari
Subject Re: Small patch to improve safety of utf8_to_unicode().
Date
Msg-id CAJTYsWUoYh84OugcnkghK24pLFsqxTy2Ajq7druCYVne_Dj8gw@mail.gmail.com
Whole thread
In response to Re: Small patch to improve safety of utf8_to_unicode().  (Jeff Davis <pgsql@j-davis.com>)
Responses Re: Small patch to improve safety of utf8_to_unicode().
List pgsql-hackers
Hi,

On Wed, 24 Jun 2026 at 11:15, Jeff Davis <pgsql@j-davis.com> wrote:
On Mon, 2026-06-22 at 19:02 -0700, Jeff Davis wrote:
> v4 attached.

v5 attached.

There's an extra patch 0002 to fix a logic bug when handling final
sigma (only affects the builtin pg_unicode_fast locale), which I think
should be backported to 18.

Also added tests.

Thanks for the patch!

I took a look at the v5 series and tried it locally. The split between the
backpatchable defensive change, the final-sigma fix, and the newer
utf8encode/utf8decode API work for master makes sense to me. 

On my machine, case-check fails before reaching the new invalid-UTF8
assertions. I think it's because PostgreSQL's Unicode tables are 17.0 while my
system ICU is 15.1, so the exhaustive ICU comparison in test_icu() hits a
changed mapping and exits first.

Would it be worth running test_convert_case() before
test_icu()? It wouldn't make case-check pass on a mismatched-ICU system, but it
would at least let the non-ICU conversion and the new invalid-UTF8 cases run
before the ICU comparison aborts.

Regards,
Ayush

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: [PATCH] Fix null pointer dereference in PG19
Next
From: shveta malik
Date:
Subject: Re: Support EXCEPT for ALL SEQUENCES publications