Re: Order changes in PG16 since ICU introduction - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: Order changes in PG16 since ICU introduction
Date
Msg-id 654a49f7ff7461bcf47be4181430678d45f93858.camel@j-davis.com
Whole thread Raw
In response to Re: Order changes in PG16 since ICU introduction  ("Daniel Verite" <daniel@manitou-mail.org>)
Responses Re: Order changes in PG16 since ICU introduction  (Jeff Davis <pgsql@j-davis.com>)
List pgsql-hackers
On Thu, 2023-04-27 at 14:23 +0200, Daniel Verite wrote:
> This should be pg_strcasecmp(...) == 0

Good catch, thank you! Fixed in updated patches.

> postgres=# create database lat9 locale 'fr_FR@euro' encoding LATIN9
> template
> 'template0';
> ERROR:  could not convert locale name "fr_FR@euro" to language tag:
> U_ILLEGAL_ARGUMENT_ERROR

ICU 63 and earlier convert it without error to the language tag 'fr-FR-
u-cu-eur', which is correct. ICU 64 removed support for transforming
some locale variants, because apparently they think those variants are
obsolete:

https://unicode-org.atlassian.net/browse/ICU-22268
https://unicode-org.atlassian.net/browse/ICU-20187

(Aside: how obsolete are those variants?)

It's frustrating that they'd remove such transformations from the
canonicalization process.

Fortunately, it looks like it's easy enough to do the transformation
ourselves. The only problematic format is '...@VARIANT'. The other
format 'fr_FR_EURO' doesn't seem to be a valid glibc locale name[1] and
windows seems to use BCP 47[2].

And there don't seem to be a lot of variants to handle. ICU 63 only
handles 3 variants, so that's what my patch does. Any unknown variant
between 5 and 8 characters won't throw an error. There could be more
problem cases, but I'm not sure how much of a practical problem they
are.

If we try to keep the meaning of LOCALE to only LC_COLLATE and
LC_CTYPE, that will continue to be confusing for anyone that uses
provider=icu.

Regards,
    Jeff Davis

[1]
https://www.gnu.org/software/libc/manual/html_node/Locale-Names.html
[2]
https://learn.microsoft.com/en-us/windows/win32/intl/locale-names

Attachment

pgsql-hackers by date:

Previous
From: Roberto Mello
Date:
Subject: Re: Postgres Version want to update from 9.2 to 9.5 version in CentOS 7.9
Next
From: John Naylor
Date:
Subject: Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound