Fixing the Turkish problem - Mailing list pgsql-hackers

From Tom Lane
Subject Fixing the Turkish problem
Date
Msg-id 26469.1083848550@sss.pgh.pa.us
Whole thread Raw
Responses Re: Fixing the Turkish problem  (Devrim GUNDUZ <devrim@gunduz.org>)
List pgsql-hackers
We're sort of halfway there on coping with the Turkish-locale i-vs-I
problem.  I'd like to finish the job for 7.5.

What we presently have is that identifier and keyword downcasing is done
without trusting tolower():
   /*    * SQL99 specifies Unicode-aware case normalization, which we don't yet    * have the infrastructure for.
Insteadwe use tolower() to provide a    * locale-aware translation.  However, there are some locales where this    * is
notright either (eg, Turkish may do strange things with 'i' and    * 'I').  Our current compromise is to use tolower()
forcharacters with    * the high bit set, and use an ASCII-only downcasing for 7-bit    * characters.    */   for (i =
0;i < len; i++)   {       unsigned char    ch = (unsigned char) ident[i];
 
       if (ch >= 'A' && ch <= 'Z')           ch += 'a' - 'A';       else if (ch >= 0x80 && isupper(ch))           ch =
tolower(ch);      result[i] = (char) ch;   }
 

AFAICS the remaining problem is that there are a bunch of places that
use strcasecmp() or strncasecmp() to match inputs against locally known
keywords (such as datestyle or timezone names).  We need to make a
variant version of strcasecmp that uses this same style of case-folding.

What I'm thinking of doing is inventing "pg_strcasecmp" and
"pg_strncasecmp" that act like the above and replacing all calls of the
standard library functions with these.

The routines need to be available in client code (eg, psql) as well as
the backend, so I'm thinking of putting them into libpgport (src/port/).
Another possibility would be to associate them with the multibyte
character code, which is already imported into client code in places.

Any thoughts, objections?
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Is there any method to keep table in memory at startup
Next
From: "Andrew Dunstan"
Date:
Subject: Re: Is there any method to keep table in memory at startup