Re: Add ASCII fast path to Unicode normalization functions - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: Add ASCII fast path to Unicode normalization functions
Date
Msg-id 77e57144-091a-4917-9c4f-6eff96096a70@dunslane.net
Whole thread
In response to Re: Add ASCII fast path to Unicode normalization functions  (Rui Zhao <zhaorui126@gmail.com>)
Responses Re: Add ASCII fast path to Unicode normalization functions
List pgsql-hackers
On 2026-10-04 Su 12:42 PM, Rui Zhao wrote:
> Hi Andrew,
>
> Thanks for working on this. I tried two further optimizations on top
> of v4. They can be applied independently:
>
> 1. unicode_assigned() still calls pg_mbstrlen_with_len() on the
> remaining input, then walks it again to check each code point. 0001
> uses the byte length to walk to the end directly. This also avoids
> scanning the whole suffix before returning false for an unassigned
> code point near the beginning.
>
> 2. unicode_normalize_func() and unicode_is_normalized() allocate a C
> string for the normalization form on every call, even when the input
> is all ASCII. 0002 compares the form directly in the text value,
> preserving case-insensitive matching and rejecting extra bytes.
> It only builds a C string when reporting an invalid form.
>
> 0003 replaces the constant-folded VALUES test with stored short and
> compressed text values.
>
> Core regression and the Unicode normalization checks passed with these
> patches.
>

OK, that all looks good. The attached combines all this.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachment

pgsql-hackers by date:

Previous
From: Greg Burd
Date:
Subject: Re: Let an ordering index scan hand its ORDER BY value to the target list
Next
From: Andrew Dunstan
Date:
Subject: Re: [PG19] COPY (query) TO ... (FORMAT json) uses the table's column names