Re: Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use. - Mailing list pgsql-hackers

Magnus Hagander wrote:
> On 25 nov 2008, at 05.00, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
>> Hiroshi Inoue <inoue@tpf.co.jp> writes:
>>> Tom Lane wrote:
>>>> If that's true then this code is presently broken for *every* locale
>>>> under Windows, not only Japanese.
>>
>>> Maybe there are a few languages/countires where 2 encodings are
>>> widely used.
>>
>> UTF8 vs Latin-N?
>
> We already special-cases utf8...
>
> I think the thing us that as long as the encodings are compatible
> (latin1 with different names for example) it worked  fine.
>
>>  In any case I think the problem is that gettext is
>> looking at a setting that is not what we are looking at.  Particularly
>> with the 8.4 changes to allow per-database locale settings, this has
>> got to be fixed in a bulletproof way.

Attached is a new patch to apply bind_textdomain_codeset() to most
server encodings. Exceptions are PG_SQL_ASCII, PG_MULE_INTERNAL
and PG_EUC_JIS_2004. "EUC-JP" may be OK for EUC_JIS_2004.

Unfortunately it's hard for Saito-san and me to check encodings
other than EUC-JP.

regards,
Hiroshi Inoue
*** mbutils.c.orig    Sun Nov 23 08:42:57 2008
--- mbutils.c    Wed Nov 26 12:17:12 2008
***************
*** 822,830 ****
--- 822,870 ----
      return clen;
  }

+ #ifdef    WIN32
+ static const struct codeset_map {
+     int    encoding;
+     const char *codeset;
+ } codeset_map_array[] = {
+         {PG_UTF8, "UTF-8"},
+         {PG_LATIN1, "LATIN1"},
+         {PG_LATIN2, "LATIN2"},
+         {PG_LATIN3, "LATIN3"},
+         {PG_LATIN4, "LATIN4"},
+         {PG_ISO_8859_5, "ISO-8859-5"},
+         {PG_ISO_8859_6, "ISO_8859-6"},
+         {PG_ISO_8859_7, "ISO-8859-7"},
+         {PG_ISO_8859_8, "ISO-8859-8"},
+         {PG_LATIN5, "LATIN5"},
+         {PG_LATIN6, "LATIN6"},
+         {PG_LATIN7, "LATIN7"},
+         {PG_LATIN8, "LATIN8"},
+         {PG_LATIN9, "LATIN-9"},
+         {PG_LATIN10, "LATIN10"},
+         {PG_KOI8R, "KOI8-R"},
+         {PG_WIN1250, "CP1250"},
+         {PG_WIN1251, "CP1251"},
+         {PG_WIN1252, "CP1252"},
+         {PG_WIN1253, "CP1253"},
+         {PG_WIN1254, "CP1254"},
+         {PG_WIN1255, "CP1255"},
+         {PG_WIN1256, "CP1256"},
+         {PG_WIN1257, "CP1257"},
+         {PG_WIN1258, "CP1258"},
+         {PG_WIN866, "CP866"},
+         {PG_WIN874, "CP874"},
+         {PG_EUC_CN, "EUC-CN"},
+         {PG_EUC_JP, "EUC-JP"},
+         {PG_EUC_KR, "EUC-KR"},
+         {PG_EUC_TW, "EUC-TW"}};
+ #endif /* WIN32 */
+
  void
  SetDatabaseEncoding(int encoding)
  {
+     const char *target_codeset = NULL;
+
      if (!PG_VALID_BE_ENCODING(encoding))
          elog(ERROR, "invalid database encoding: %d", encoding);

***************
*** 846,852 ****
       */
  #ifdef ENABLE_NLS
      if (encoding == PG_UTF8)
!         if (bind_textdomain_codeset("postgres", "UTF-8") == NULL)
              elog(LOG, "bind_textdomain_codeset failed");
  #endif
  }
--- 886,907 ----
       */
  #ifdef ENABLE_NLS
      if (encoding == PG_UTF8)
!         target_codeset = "UTF-8";
! #ifdef    WIN32
!     else
!     {
!         int    i;
!
!         for (i = 0; i < sizeof(codeset_map_array) / sizeof(struct codeset_map); i++)
!             if (codeset_map_array[i].encoding == encoding)
!             {
!                 target_codeset = codeset_map_array[i].codeset;
!                 break;
!             }
!     }
! #endif /* WIN32 */
!     if (target_codeset != NULL)
!         if (bind_textdomain_codeset("postgres", target_codeset) == NULL)
              elog(LOG, "bind_textdomain_codeset failed");
  #endif
  }