Re: UPPER()/LOWER() and UTF-8 - Mailing list pgsql-hackers

From Tom Lane
Subject Re: UPPER()/LOWER() and UTF-8
Date
Msg-id 1701.1068043341@sss.pgh.pa.us
Whole thread Raw
In response to UPPER()/LOWER() and UTF-8  (Alexey Mahotkin <alexm@w-m.ru>)
Responses Re: UPPER()/LOWER() and UTF-8
List pgsql-hackers
Alexey Mahotkin <alexm@w-m.ru> writes:
>     TL> upper/lower aren't going to work desirably in any multi-byte
>     TL> character set encoding.  

> Can you please point me at their implementation?  I do not understand
> why that's impossible.

Because they use <ctype.h>'s toupper() and tolower() functions, which
only work on single-byte characters.

There has been some discussion of using <wctype.h> where available, but
this has a number of issues, notably figuring out the correct mapping
from the server string encoding (eg UTF-8) to unpacked wide characters.
At minimum we'd need to know which charset the locale setting is
expecting, and there doesn't seem to be a portable way to find that out.

IIRC, Peter thinks we must abandon use of libc's locale functionality
altogether and write our own locale layer before we can really have all
the locale-specific functionality we want.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Andreas Pflug
Date:
Subject: Re: Open Sourcing pgManage
Next
From: Tom Lane
Date:
Subject: Re: weird regression test issue CVS HEAD