Re: Unicode restriction - Mailing list pgsql-hackers

From Tatsuo Ishii
Subject Re: Unicode restriction
Date
Msg-id 20040803.215933.78705094.t-ishii@sra.co.jp
Whole thread Raw
In response to Unicode restriction  (Oliver Elphick <olly@lfix.co.uk>)
Responses Re: Unicode restriction  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
> In src/backend/utils/mb/wchar.c there is a check to exclude Unicode
> characters above 0x10000.  I can't see anything to explain this
> restriction, except possibly this in the release notes for 7.2:
> 
>         Reject invalid multibyte character sequences (Tatsuo)
> 
> It does not explain why part of the Unicode character range is invalid. 
> There is a Debian bug report from someone whose client is trying to
> store characters in the excluded range.  What would be needed to enable
> support for it?

Before 7.4, to be handled by regex routines, UTF-8 are converted to
ISO 10646. There was a limitaion in regex routines in that they cannot
handle multibyte characters > 2bytes. In another word only 16bit UCS-2
are supported. That's why ISO 10646 > 0x10000 is rejected.

I'm not sure if the regex routines include in 7.4 or later has this
restrictions or not. If not, probably we could remove the check (with
losing data compatibilty).
--
Tatsuo Ishii


pgsql-hackers by date:

Previous
From: Gavin Sherry
Date:
Subject: Re: Anybody have an Oracle PL/SQL reference at hand?
Next
From: "Greg Sabino Mullane"
Date:
Subject: Re: Open items