Re: lower and upper not UTF-8 safe - Mailing list pgsql-hackers

From Karel Zak
Subject Re: lower and upper not UTF-8 safe
Date
Msg-id 20030805065850.GA12563@zf.jcu.cz
Whole thread Raw
In response to Re: lower and upper not UTF-8 safe  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: lower and upper not UTF-8 safe
List pgsql-hackers
On Mon, Aug 04, 2003 at 05:03:02PM -0400, Tom Lane wrote:
> Julian Satchell <j.satchell@eris.qinetiq.com> writes:
> > The implementations of lower and upper in
> > src/backend/utils/adt/oracle_compat.c use the single byte macros from
> > ctype.h to alter individual bytes in the text string. 
> 
> > If the text is UTF-8 encoded this is totally wrong, and will result in
> > an invalid string that is no longer UTF-8.
> 
> Only if you use a locale that is assuming a character set that is not
> UTF8 but does have characters with the high bit set.  I'm not sure that
> we can do anything to defend against locale/charset mismatch.
We can try detect typical locale charset and compare it with actualcharset used in DB and send NOTICE to FE if it's
mismatched.The problem is portability of charset detection code, because there is differences between OS. The best it's
iflibc support nl_langinfo(CODESET) call.The complete code of charset detection you can found in libcharset orglib (I
usesimplification of these codes and it's 300 lines:-).
 
   Karel


-- Karel Zak  <zakkr@zf.jcu.cz>http://home.zf.jcu.cz/~zakkr/


pgsql-hackers by date:

Previous
From: Oleg Bartunov
Date:
Subject: Re: Release changes
Next
From: "Shridhar Daithankar"
Date:
Subject: 7.4 beta binaries