Thread: Re: [BUGS] BUG #4186: set lc_messages does not work
Hi, I posted a patch 18 days ago but have got no responce. Anyway I've simplified the patch by using an appropriate gettext module. Hiroshi Inoue wrote: > Bruce Momjian wrote: >> Tom Lane wrote: >>> Magnus Hagander <magnus@hagander.net> writes: >>>> Thomas H. wrote: >>>>> so at least that explains the "changed" behaviour. nevertheless, >>>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>>> pg always picks the os' language and ignores the lc_message value. >>>> This looks like I can reproduce though, at least on cvs head. Did this >>>> work for you in previous versions? >>> Maybe we were using a different build of gettext in the previous >>> releases, one that didn't look at the same info as the current code? >>> >> >> Where are we on this? AFAICS there are 2 causes. 1. MSVC version of postgres is using a bad gettext module. 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. As for 1, we have to use another gettext module. I can provide it if requested. As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls corresponding putenv() whose result can be referenced by getenv() in mingw. In addtion the patch provides a functionality to Windows locale name to ISO formatted locale name. Comments ? regards, Hiroshi Inoue
Oops, I forgot to attach the patch, sorry.
Hiroshi Inoue wrote:
> Hi,
>
> I posted a patch 18 days ago but have got no responce.
> Anyway I've simplified the patch by using an appropriate
> gettext module.
>
> Hiroshi Inoue wrote:
>> Bruce Momjian wrote:
>>> Tom Lane wrote:
>>>> Magnus Hagander <magnus@hagander.net> writes:
>>>>> Thomas H. wrote:
>>>>>> so at least that explains the "changed" behaviour. nevertheless,
>>>>>> LC_MESSAGES seems to be defunct - with the "locale" folder present,
>>>>>> pg always picks the os' language and ignores the lc_message value.
>>>>> This looks like I can reproduce though, at least on cvs head. Did this
>>>>> work for you in previous versions?
>>>> Maybe we were using a different build of gettext in the previous
>>>> releases, one that didn't look at the same info as the current code?
>>>>
>>>
>>> Where are we on this?
>
> AFAICS there are 2 causes.
>
> 1. MSVC version of postgres is using a bad gettext module.
> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0.
>
> As for 1, we have to use another gettext module. I can provide it
> if requested.
> As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls
> corresponding putenv() whose result can be referenced by getenv() in
> mingw.
>
> In addtion the patch provides a functionality to Windows locale
> name to ISO formatted locale name.
>
> Comments ?
>
> regards,
> Hiroshi Inoue
Index: backend/main/main.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/main/main.c,v
retrieving revision 1.111
diff -c -c -r1.111 main.c
*** backend/main/main.c 11 Dec 2008 07:34:07 -0000 1.111
--- backend/main/main.c 26 Dec 2008 11:54:30 -0000
***************
*** 132,138 ****
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! unsetenv("LC_ALL");
/*
* Catch standard options before doing much else
--- 132,138 ----
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! pg_unsetenv("LC_ALL");
/*
* Catch standard options before doing much else
Index: backend/utils/adt/pg_locale.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v
retrieving revision 1.42
diff -c -c -r1.42 pg_locale.c
*** backend/utils/adt/pg_locale.c 23 Sep 2008 09:20:36 -0000 1.42
--- backend/utils/adt/pg_locale.c 26 Dec 2008 11:54:31 -0000
***************
*** 88,93 ****
--- 88,102 ----
static char lc_numeric_envbuf[LC_ENV_BUFSIZE];
static char lc_time_envbuf[LC_ENV_BUFSIZE];
+ /*
+ * The following 2 functions are ajusted ones for Windows MSVC.
+ */
+ static int pg_putenv(const char *); /* putenv() adjusted for MSVC */
+ extern void pg_unsetenv(const char *); /* unsetenv() adjusted for MSVC */
+ /* The following function is available under MSVC8.0 or later */
+ #ifdef WIN32
+ static char *IsoLocaleName(const char *); /* MSVC specific */
+ #endif
/*
* pg_perm_setlocale
***************
*** 147,152 ****
--- 156,167 ----
case LC_MESSAGES:
envvar = "LC_MESSAGES";
envbuf = lc_messages_envbuf;
+ #ifdef WIN32
+ if (result = IsoLocaleName(locale), NULL == result)
+ result = locale;
+ #endif /* WIN32 */
break;
#endif
case LC_MONETARY:
***************
*** 165,170 ****
--- 180,186 ----
elog(FATAL, "unrecognized LC category: %d", category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
+ return NULL;
break;
}
***************
*** 181,189 ****
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (_putenv(envbuf))
! return NULL;
! #endif
return result;
}
--- 197,205 ----
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (pg_putenv(envbuf))
! result = NULL;
! #endif /* WIN32 */
return result;
}
***************
*** 214,219 ****
--- 230,335 ----
return ret;
}
+ /*
+ * Ajusted version of putenv() for MSVC.
+ */
+ static
+ int pg_putenv(const char *envval)
+ {
+ #if defined(_MSC_VER) && (_MSC_VER >= 1300) /* VC7.0 or later */
+ /*
+ * Each MSVC version has its own _putenv() in its runtime library
+ * msvcrXX.dll. So we have to call _putenv() in msvcrt.dll so as
+ * to be referenced by GnuWin32 library.
+ */
+ typedef int (_cdecl *PUTENVPROC)(const char *);
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+ int ret;
+
+ if (NULL == putenvFunc)
+ {
+ if (hmodule = GetModuleHandle("msvcrt"), NULL == hmodule)
+ return 1;
+ putenvFunc = (PUTENVPROC)GetProcAddress(hmodule, "_putenv");
+ }
+ if (NULL == putenvFunc)
+ return 1;
+ if (ret = putenvFunc(envval), 0 != ret)
+ return ret;
+ #endif /* _MSC_VER >= 1300 */
+ return putenv(envval);
+ }
+
+ /*
+ * Adjusted version of unsetenv() for MSVC.
+ */
+ void pg_unsetenv(const char *name)
+ {
+ #ifdef WIN32
+ char *envbuf;
+
+ if (envbuf = (char *) malloc(strlen(name) + 2))
+ {
+ sprintf(envbuf, "%s=", name);
+ pg_putenv(envbuf);
+ free(envbuf);
+ }
+ #else
+ unsetenv(name);
+ #endif /* WIN32 */
+ }
+
+ #ifdef WIN32
+ /*
+ * Convert Windows locale name to the IS formatted one
+ * if possible.
+ *
+ * This function returns NULL if conversion is impossible,
+ * otherwise returns the pointer to a static area which
+ * contains the iso formatted locale name.
+ */
+
+ static
+ char *IsoLocaleName(const char *winlocname)
+ {
+ #if (_MSC_VER >= 1400) /* VC8.0 or later */
+ #include <shlwapi.h>
+
+ static char iso_lc_messages[32];
+ int usecategory = LC_CTYPE;
+ _locale_t loct = NULL;
+
+ if (0 == stricmp("c", winlocname) ||
+ 0 == stricmp("posix", winlocname))
+ {
+ strncpy(iso_lc_messages, "C", sizeof(iso_lc_messages));
+ return iso_lc_messages;
+ }
+
+ loct = _create_locale(usecategory, winlocname);
+ if (NULL != loct)
+ {
+ char isolang[32], isocrty[32];
+ LCID lcid;
+
+ lcid = loct->locinfo->lc_handle[usecategory];
+ if (0 == lcid)
+ lcid = MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT);
+ _free_locale(loct);
+
+
+ GetLocaleInfoA(lcid, LOCALE_SISO639LANGNAME, isolang, sizeof(isolang));
+ GetLocaleInfoA(lcid, LOCALE_SISO3166CTRYNAME, isocrty, sizeof(isocrty));
+ snprintf(iso_lc_messages, sizeof(iso_lc_messages) - 1, "%s_%s", isolang, isocrty);
+ return iso_lc_messages;
+ }
+ return NULL;
+ #else
+ return NULL; /* does nothing. */
+ #endif /* _MSC_VER >= 1400 */
+ }
+ #endif /* WIN32 */
+
/* GUC assign hooks */
/*
Index: include/utils/pg_locale.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/utils/pg_locale.h,v
retrieving revision 1.26
diff -c -c -r1.26 pg_locale.h
*** include/utils/pg_locale.h 23 Sep 2008 09:20:39 -0000 1.26
--- include/utils/pg_locale.h 26 Dec 2008 11:54:34 -0000
***************
*** 53,56 ****
--- 53,58 ----
extern void cache_locale_time(void);
+ extern void pg_unsetenv(const char *);
+
#endif /* _PG_LOCALE_ */
Hi. Sorry very late reaction.... I report the test checked again. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES_01.png http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES_02.png http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES_03.png http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES_04.png and http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES.sql Then, set PGCLIENTENCODING=SJIS http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/LC_MESSAGES.log However, libintl needs to be created correctly.(Inoue-san prepares this.) http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/bin/ http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/include/ http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/lib/ in CVS-HEAD of the newest patch, line positions differ for a while.(.Inoue-san prepares this.) http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/mbutils_2.patch http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/win_lc_messages_2.patch Conclusion, looks at a good result.!! Regards, Hiroshi Saito ----- Original Message ----- From: "Hiroshi Inoue" <inoue@tpf.co.jp> > Oops, I forgot to attach the patch, sorry. > > Hiroshi Inoue wrote: >> Hi, >> >> I posted a patch 18 days ago but have got no responce. >> Anyway I've simplified the patch by using an appropriate >> gettext module. >> >> Hiroshi Inoue wrote: >>> Bruce Momjian wrote: >>>> Tom Lane wrote: >>>>> Magnus Hagander <magnus@hagander.net> writes: >>>>>> Thomas H. wrote: >>>>>>> so at least that explains the "changed" behaviour. nevertheless, >>>>>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>>>>> pg always picks the os' language and ignores the lc_message value. >>>>>> This looks like I can reproduce though, at least on cvs head. Did this >>>>>> work for you in previous versions? >>>>> Maybe we were using a different build of gettext in the previous >>>>> releases, one that didn't look at the same info as the current code? >>>>> >>>> >>>> Where are we on this? >> >> AFAICS there are 2 causes. >> >> 1. MSVC version of postgres is using a bad gettext module. >> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. >> >> As for 1, we have to use another gettext module. I can provide it >> if requested. >> As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls >> corresponding putenv() whose result can be referenced by getenv() in >> mingw. >> >> In addtion the patch provides a functionality to Windows locale >> name to ISO formatted locale name. >> >> Comments ? >> >> regards, >> Hiroshi Inoue >
Hiroshi Inoue wrote: > Hiroshi Inoue wrote: >> Hi, >> >> I posted a patch 18 days ago but have got no responce. >> Anyway I've simplified the patch by using an appropriate >> gettext module. >> >> Hiroshi Inoue wrote: >>> Bruce Momjian wrote: >>>> Tom Lane wrote: >>>>> Magnus Hagander <magnus@hagander.net> writes: >>>>>> Thomas H. wrote: >>>>>>> so at least that explains the "changed" behaviour. nevertheless, >>>>>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>>>>> pg always picks the os' language and ignores the lc_message value. >>>>>> This looks like I can reproduce though, at least on cvs head. Did >>>>>> this >>>>>> work for you in previous versions? >>>>> Maybe we were using a different build of gettext in the previous >>>>> releases, one that didn't look at the same info as the current code? >>>>> >>>> >>>> Where are we on this? >> >> AFAICS there are 2 causes. >> >> 1. MSVC version of postgres is using a bad gettext module. >> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. >> >> As for 1, we have to use another gettext module. I can provide it >> if requested. Yes, I think that'll be needed. Exactly what is wrong and needs to be changed? (Copying DAve in on this since he builds the MSI) Is it possible to build this one with the same version of MSVC? If it is, then that should remove the need for #2, right? >> As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls >> corresponding putenv() whose result can be referenced by getenv() in >> mingw. Oh, yuck. This must be because msvcrt.dll (used by mingw) caches the values in the environment. That's a rather ugly solution, but I guess if we have no other choice.. Does it make a difference if you try to set the value using SetEnvironmentVariable()? It would definitely work if the gettext stuff uses GetEnvironmentVariable(). I doubt it does though, but it might work anyway... >> In addtion the patch provides a functionality to Windows locale >> name to ISO formatted locale name. >> >> Comments ? I wonder if it's cleaner to use this "load msvcrt version of setenv()" *always*. Or to cover all bases, perhaps we should always do *both* - that is, both set in current runtime and msvcrt.dll... We don't do this in a lot of places today, but we might use more in the future. And as long as it's not in a performance critical path, doing it twice is almost for free... There is already src/port/unsetenv.c, we should probably be building on top of that one. As for the code itself, there are some very non-postgresql-standard constructs in the patch, like: + if (result = IsoLocaleName(locale), NULL == result) + result = locale; And things like: + return NULL; break; is just wrong. And we don't normally write if (NULL == <something>). But these are fairly simple to fix, so I can do those at commit-time. //Magnus
Magnus Hagander wrote: > Hiroshi Inoue wrote: >> Hiroshi Inoue wrote: >>>>> Where are we on this? >>> AFAICS there are 2 causes. >>> >>> 1. MSVC version of postgres is using a bad gettext module. >>> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. >>> >>> As for 1, we have to use another gettext module. I can provide it >>> if requested. > > Yes, I think that'll be needed. Exactly what is wrong and needs to be > changed? (Copying DAve in on this since he builds the MSI) As far as I see, the libintl3.dll(version 0.14.4.1952) at http://sourceforge.net/project/showfiles.php?group_id=23617&package_id=17255&release_id=325693 was built under a bad configuration. In fact I can see HAVE_LC_MESSAGES #defined in src\gettext\0.14.4\gettext-0.14.4\gettext-runtime\config.h. Probably due to this mistake, the gettext module libint3.dll doesn't see the environment variables LC_MESSAGES etc at alland tries to call setlocale() instead. > Is it possible to build this one with the same version of MSVC? If it > is, then that should remove the need for #2, right? > >>> As for 2, pg_putenv() or pg_unsetenv() in the attachced patch calls >>> corresponding putenv() whose result can be referenced by getenv() in >>> mingw. > > Oh, yuck. This must be because msvcrt.dll (used by mingw) caches the > values in the environment. > > That's a rather ugly solution, but I guess if we have no other choice.. > Does it make a difference if you try to set the value using > SetEnvironmentVariable()? SetEnvironmentVariable() is called from the first in pg_perm_setlocale(). > It would definitely work if the gettext stuff uses > GetEnvironmentVariable(). Yes I think so. > I doubt it does though, but it might work > anyway... > > >>> In addtion the patch provides a functionality to Windows locale >>> name to ISO formatted locale name. >>> >>> Comments ? > > I wonder if it's cleaner to use this "load msvcrt version of setenv()" > *always*. Or to cover all bases, perhaps we should always do *both* - > that is, both set in current runtime and msvcrt.dll... We don't do this > in a lot of places today, but we might use more in the future. And as > long as it's not in a performance critical path, doing it twice is > almost for free... Agreed. regards, Hiroshi Inoue
Magnus Hagander wrote:
> Hiroshi Inoue wrote:
>>> AFAICS there are 2 causes.
>>>
>>> 1. MSVC version of postgres is using a bad gettext module.
>>> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0.
>>>
>>> As for 1, we have to use another gettext module. I can provide it
>>> if requested.
>
> Yes, I think that'll be needed. Exactly what is wrong and needs to be
> changed? (Copying DAve in on this since he builds the MSI)
>
> Is it possible to build this one with the same version of MSVC? If it
> is, then that should remove the need for #2, right?
Maybe so but what I built was gettext-0.17 and the README.woe32 says
This file explains how to create binaries for the mingw
execution environment..... MS Visual C/C++ with "nmake" is no longer supported.
BTW gettext-0.14.4 has both README.mingw and README.woe32. REAME.woe32
seems to be for MSVC build.
regards,
Hiroshi Inoue
Hiroshi Inoue wrote: > Magnus Hagander wrote: >> Hiroshi Inoue wrote: >>>> AFAICS there are 2 causes. >>>> >>>> 1. MSVC version of postgres is using a bad gettext module. >>>> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. >>>> >>>> As for 1, we have to use another gettext module. I can provide it >>>> if requested. >> >> Yes, I think that'll be needed. Exactly what is wrong and needs to be >> changed? (Copying DAve in on this since he builds the MSI) >> >> Is it possible to build this one with the same version of MSVC? If it >> is, then that should remove the need for #2, right? > > Maybe so but what I built was gettext-0.17 and the README.woe32 says > > This file explains how to create binaries for the mingw > execution environment..... > MS Visual C/C++ with "nmake" is no longer supported. > > BTW gettext-0.14.4 has both README.mingw and README.woe32. REAME.woe32 > seems to be for MSVC build. They are so annoying... Anyway, if they don't support it, we shuldn't try to force it. Let's go down the path that they will have different ones, and make getenv() and friends update both. Do you want to send an updated patch for it, or do you want me to look at it? //Magus
Magnus Hagander wrote: > Hiroshi Inoue wrote: >> Magnus Hagander wrote: >>> Hiroshi Inoue wrote: >>>>> AFAICS there are 2 causes. >>>>> >>>>> 1. MSVC version of postgres is using a bad gettext module. >>>>> 2. getenv() in mingw cannot see the result of putenv() in MSVC8.0. >>>>> >>>>> As for 1, we have to use another gettext module. I can provide it >>>>> if requested. >>> Yes, I think that'll be needed. Exactly what is wrong and needs to be >>> changed? (Copying DAve in on this since he builds the MSI) >>> >>> Is it possible to build this one with the same version of MSVC? If it >>> is, then that should remove the need for #2, right? >> Maybe so but what I built was gettext-0.17 and the README.woe32 says >> >> This file explains how to create binaries for the mingw >> execution environment..... >> MS Visual C/C++ with "nmake" is no longer supported. >> >> BTW gettext-0.14.4 has both README.mingw and README.woe32. REAME.woe32 >> seems to be for MSVC build. > > They are so annoying... > > Anyway, if they don't support it, we shuldn't try to force it. Let's go > down the path that they will have different ones, and make getenv() and > friends update both. > > Do you want to send an updated patch for it, or do you want me to look > at it? I would send a new patch to which I added a simple ISO style check for locale names. regards, Hiroshi Inoue
Hiroshi Inoue wrote:
> Magnus Hagander wrote:
>>
>> Do you want to send an updated patch for it, or do you want me to look
>> at it?
>
> I would send a new patch to which I added a simple ISO style check for
> locale names.
Attached is a new patch.
I added a simple ISO style locale name check.
Avoided codings like *NULL == somthing*.
It also includes the changes to mbutils.c and elog.c which fix
recently introduced bug by the domain name change from "postgres"
to "postgres-8.4".
regards,
Hiroshi Inoue
Index: backend/utils/adt/pg_locale.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v
retrieving revision 1.43
diff -c -c -r1.43 pg_locale.c
*** backend/utils/adt/pg_locale.c 1 Jan 2009 17:23:49 -0000 1.43
--- backend/utils/adt/pg_locale.c 8 Jan 2009 19:26:02 -0000
***************
*** 88,93 ****
--- 88,102 ----
static char lc_numeric_envbuf[LC_ENV_BUFSIZE];
static char lc_time_envbuf[LC_ENV_BUFSIZE];
+ /*
+ * The following 2 functions are ajusted ones for Windows MSVC.
+ */
+ static int pg_putenv(const char *); /* putenv() adjusted for MSVC */
+ extern void pg_unsetenv(const char *); /* unsetenv() adjusted for MSVC */
+ /* The following function is available under MSVC8.0 or later */
+ #ifdef WIN32
+ static char *IsoLocaleName(const char *); /* MSVC specific */
+ #endif
/*
* pg_perm_setlocale
***************
*** 147,152 ****
--- 156,167 ----
case LC_MESSAGES:
envvar = "LC_MESSAGES";
envbuf = lc_messages_envbuf;
+ /* Refresh msgid pool maintained by gettext? */
+ // textdomain(textdomain(NULL));
+ #ifdef WIN32
+ if (result = IsoLocaleName(locale), result == NULL)
+ return NULL;
+ #endif /* WIN32 */
break;
#endif
case LC_MONETARY:
***************
*** 165,171 ****
elog(FATAL, "unrecognized LC category: %d", category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! break;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, "%s=%s", envvar, result);
--- 180,186 ----
elog(FATAL, "unrecognized LC category: %d", category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! return NULL;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, "%s=%s", envvar, result);
***************
*** 181,189 ****
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (_putenv(envbuf))
! return NULL;
! #endif
return result;
}
--- 196,204 ----
*/
if (!SetEnvironmentVariable(envvar, result))
return NULL;
! if (pg_putenv(envbuf))
! result = NULL;
! #endif /* WIN32 */
return result;
}
***************
*** 214,219 ****
--- 229,352 ----
return ret;
}
+ /*
+ * Ajusted version of putenv() for MSVC.
+ */
+ static
+ int pg_putenv(const char *envval)
+ {
+ #if defined(_MSC_VER) && (_MSC_VER >= 1300) /* VC7.0 or later */
+ /*
+ * Each MSVC version has its own _putenv() in its runtime library
+ * msvcrXX.dll. So we have to call _putenv() in msvcrt.dll so as
+ * to be referenced by GnuWin32 library.
+ */
+ typedef int (_cdecl *PUTENVPROC)(const char *);
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+ int ret;
+
+ if (putenvFunc == NULL)
+ {
+ if (hmodule = GetModuleHandle("msvcrt"), hmodule == NULL)
+ return 1;
+ putenvFunc = (PUTENVPROC) GetProcAddress(hmodule, "_putenv");
+ if (putenvFunc == NULL)
+ return 1;
+ }
+ if (ret = putenvFunc(envval), ret != 0)
+ return ret;
+ #endif /* _MSC_VER >= 1300 */
+ return putenv(envval);
+ }
+
+ /*
+ * Adjusted version of unsetenv() for MSVC.
+ */
+ void pg_unsetenv(const char *name)
+ {
+ #ifdef WIN32
+ char *envbuf;
+
+ if (envbuf = (char *) malloc(strlen(name) + 2))
+ {
+ sprintf(envbuf, "%s=", name);
+ pg_putenv(envbuf);
+ free(envbuf);
+ }
+ #else
+ unsetenv(name);
+ #endif /* WIN32 */
+ }
+
+ #ifdef WIN32
+ /*
+ * Convert Windows locale name to the ISO's one if possible.
+ *
+ * This function returns NULL if conversion is impossible
+ * and the format style isn't ISO, otherwise returns the
+ * ISO formatted locale name.
+ */
+ static
+ char *IsoLocaleName(const char *winlocname)
+ {
+ #if (_MSC_VER >= 1400) /* VC8.0 or later */
+ static char iso_lc_messages[32];
+ int usecategory = LC_CTYPE;
+ _locale_t loct = NULL;
+
+ if (stricmp("c", winlocname) == 0 ||
+ stricmp("posix", winlocname) == 0)
+ {
+ strncpy(iso_lc_messages, "C", sizeof(iso_lc_messages));
+ return iso_lc_messages;
+ }
+
+ loct = _create_locale(usecategory, winlocname);
+ if (loct != NULL)
+ {
+ char isolang[32], isocrty[32];
+ LCID lcid;
+
+ lcid = loct->locinfo->lc_handle[usecategory];
+ if (lcid == 0)
+ lcid = MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT);
+ _free_locale(loct);
+
+
+ GetLocaleInfoA(lcid, LOCALE_SISO639LANGNAME, isolang, sizeof(isolang));
+ GetLocaleInfoA(lcid, LOCALE_SISO3166CTRYNAME, isocrty, sizeof(isocrty));
+ snprintf(iso_lc_messages, sizeof(iso_lc_messages) - 1, "%s_%s", isolang, isocrty);
+ return iso_lc_messages;
+ }
+ else /* ISO style check */
+ {
+ int lclen;
+ char *dotp;
+
+ if (dotp = strchr(winlocname, '.'), dotp != NULL)
+ lclen = dotp - winlocname;
+ else
+ lclen = strlen(winlocname);
+ switch (lclen)
+ {
+ case 2:
+ break;
+ case 5:
+ if (winlocname[2] != '_')
+ return NULL;
+ break;
+ default:
+ return NULL;
+ }
+ }
+ return winlocname;
+ #else
+ return winlocname; /* does nothing. */
+ #endif /* _MSC_VER >= 1400 */
+ }
+ #endif /* WIN32 */
+
/* GUC assign hooks */
/*
Index: backend/main/main.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/main/main.c,v
retrieving revision 1.112
diff -c -c -r1.112 main.c
*** backend/main/main.c 1 Jan 2009 17:23:43 -0000 1.112
--- backend/main/main.c 8 Jan 2009 19:26:03 -0000
***************
*** 132,138 ****
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! unsetenv("LC_ALL");
/*
* Catch standard options before doing much else
--- 132,138 ----
* environment, remove any LC_ALL setting, so that the environment
* variables installed by pg_perm_setlocale have force.
*/
! pg_unsetenv("LC_ALL");
/*
* Catch standard options before doing much else
Index: include/utils/pg_locale.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/utils/pg_locale.h,v
retrieving revision 1.27
diff -c -c -r1.27 pg_locale.h
*** include/utils/pg_locale.h 1 Jan 2009 17:24:02 -0000 1.27
--- include/utils/pg_locale.h 8 Jan 2009 19:26:03 -0000
***************
*** 53,56 ****
--- 53,58 ----
extern void cache_locale_time(void);
+ extern void pg_unsetenv(const char *);
+
#endif /* _PG_LOCALE_ */
Index: backend/utils/mb/mbutils.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/mb/mbutils.c,v
retrieving revision 1.76
diff -c -c -r1.76 mbutils.c
*** backend/utils/mb/mbutils.c 4 Jan 2009 18:37:35 -0000 1.76
--- backend/utils/mb/mbutils.c 8 Jan 2009 19:26:03 -0000
***************
*** 873,879 ****
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! if (bind_textdomain_codeset("postgres", "UTF-8") == NULL)
elog(LOG, "bind_textdomain_codeset failed");
#endif
}
--- 873,879 ----
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! if (bind_textdomain_codeset(textdomain(NULL), "UTF-8") == NULL)
elog(LOG, "bind_textdomain_codeset failed");
#endif
}
Index: backend/utils/error/elog.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/error/elog.c,v
retrieving revision 1.211
diff -c -c -r1.211 elog.c
*** backend/utils/error/elog.c 7 Jan 2009 04:26:46 -0000 1.211
--- backend/utils/error/elog.c 8 Jan 2009 19:26:05 -0000
***************
*** 308,314 ****
edata->lineno = lineno;
edata->funcname = funcname;
/* the default text domain is the backend's */
! edata->domain = domain ? domain : "postgres";
/* Select default errcode based on elevel */
if (elevel >= ERROR)
edata->sqlerrcode = ERRCODE_INTERNAL_ERROR;
--- 308,314 ----
edata->lineno = lineno;
edata->funcname = funcname;
/* the default text domain is the backend's */
! edata->domain = domain ? domain : PG_TEXTDOMAIN("postgres");
/* Select default errcode based on elevel */
if (elevel >= ERROR)
edata->sqlerrcode = ERRCODE_INTERNAL_ERROR;
Hiroshi Inoue wrote:
> Hiroshi Inoue wrote:
>> Magnus Hagander wrote:
>>>
>>> Do you want to send an updated patch for it, or do you want me to look
>>> at it?
>>
>> I would send a new patch to which I added a simple ISO style check for
>> locale names.
>
> Attached is a new patch.
> I added a simple ISO style locale name check.
> Avoided codings like *NULL == somthing*.
> It also includes the changes to mbutils.c and elog.c which fix
> recently introduced bug by the domain name change from "postgres"
> to "postgres-8.4".
Attached is a further updated version of this patch. Changes include:
* Actually avoid NULL==something style coding everywhere (I think)
* Avoid coding like "if (ret = putenvFunc(envval), 0 != ret)"
* Per discussion, remove pg_locale.c specific coding, put it in
src/port, and do a redefine so we *always* use these new functions
* Some further minor cleanups
* Removed the change to mbutils.c and elog.c - those are a separate
issue, will deal with those as a separate patch.
There still needs to be some error checking added in IsoLocaleName(),
but this is a start.
Can someone please test this? :-) I can't get NLS to work at all in my
Windows install (this is nothing new, it's always been tricky) right
now. I'll work on trying to fix that, but help with testing would be
very useful meanwhile.
(It passes build on MSVC, it's just runtime i can't check)
//Magnus
*** a/configure
--- b/configure
***************
*** 17920,17925 **** case " $LIBOBJS " in
--- 17920,17931 ----
esac
case " $LIBOBJS " in
+ *" win32env.$ac_objext "* ) ;;
+ *) LIBOBJS="$LIBOBJS win32env.$ac_objext"
+ ;;
+ esac
+
+ case " $LIBOBJS " in
*" win32error.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS win32error.$ac_objext"
;;
*** a/configure.in
--- b/configure.in
***************
*** 1274,1279 **** AC_REPLACE_FUNCS(gettimeofday)
--- 1274,1280 ----
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
AC_LIBOBJ(rand)
+ AC_LIBOBJ(win32env)
AC_LIBOBJ(win32error)
AC_DEFINE([HAVE_SYMLINK], 1,
[Define to 1 if you have the `symlink' function.])
*** a/src/backend/utils/adt/pg_locale.c
--- b/src/backend/utils/adt/pg_locale.c
***************
*** 55,60 ****
--- 55,63 ----
#include "utils/memutils.h"
#include "utils/pg_locale.h"
+ #ifdef WIN32
+ #include <shlwapi.h>
+ #endif
#define MAX_L10N_DATA 80
***************
*** 89,94 **** static char lc_monetary_envbuf[LC_ENV_BUFSIZE];
--- 92,101 ----
static char lc_numeric_envbuf[LC_ENV_BUFSIZE];
static char lc_time_envbuf[LC_ENV_BUFSIZE];
+ #ifdef WIN32
+ static char *IsoLocaleName(const char *); /* MSVC specific */
+ #endif
+
/*
* pg_perm_setlocale
***************
*** 148,155 **** pg_perm_setlocale(int category, const char *locale)
case LC_MESSAGES:
envvar = "LC_MESSAGES";
envbuf = lc_messages_envbuf;
break;
! #endif
case LC_MONETARY:
envvar = "LC_MONETARY";
envbuf = lc_monetary_envbuf;
--- 155,167 ----
case LC_MESSAGES:
envvar = "LC_MESSAGES";
envbuf = lc_messages_envbuf;
+ #ifdef WIN32
+ result = IsoLocaleName(locale);
+ if (result == NULL)
+ result = locale;
+ #endif /* WIN32 */
break;
! #endif /* LC_MESSAGES */
case LC_MONETARY:
envvar = "LC_MONETARY";
envbuf = lc_monetary_envbuf;
***************
*** 166,190 **** pg_perm_setlocale(int category, const char *locale)
elog(FATAL, "unrecognized LC category: %d", category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! break;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, "%s=%s", envvar, result);
- #ifndef WIN32
if (putenv(envbuf))
return NULL;
- #else
-
- /*
- * On Windows, we need to modify both the process environment and the
- * cached version in msvcrt
- */
- if (!SetEnvironmentVariable(envvar, result))
- return NULL;
- if (_putenv(envbuf))
- return NULL;
- #endif
return result;
}
--- 178,190 ----
elog(FATAL, "unrecognized LC category: %d", category);
envvar = NULL; /* keep compiler quiet */
envbuf = NULL;
! return NULL;
}
snprintf(envbuf, LC_ENV_BUFSIZE - 1, "%s=%s", envvar, result);
if (putenv(envbuf))
return NULL;
return result;
}
***************
*** 599,601 **** cache_locale_time(void)
--- 599,650 ----
CurrentLCTimeValid = true;
}
+
+
+ #ifdef WIN32
+ /*
+ * Convert Windows locale name to the ISO formatted one
+ * if possible.
+ *
+ * This function returns NULL if conversion is impossible,
+ * otherwise returns the pointer to a static area which
+ * contains the iso formatted locale name.
+ */
+ static
+ char *IsoLocaleName(const char *winlocname)
+ {
+ #if (_MSC_VER >= 1400) /* VC8.0 or later */
+
+ static char iso_lc_messages[32];
+ _locale_t loct = NULL;
+
+ if (pg_strcasecmp("c", winlocname) == 0 ||
+ pg_strcasecmp("posix", winlocname) == 0)
+ {
+ strncpy(iso_lc_messages, "C", sizeof(iso_lc_messages));
+ return iso_lc_messages;
+ }
+
+ loct = _create_locale(LC_CTYPE, winlocname);
+ if (loct != NULL)
+ {
+ char isolang[32], isocrty[32];
+ LCID lcid;
+
+ lcid = loct->locinfo->lc_handle[LC_CTYPE];
+ if (lcid == 0)
+ lcid = MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT);
+ _free_locale(loct);
+
+ GetLocaleInfoA(lcid, LOCALE_SISO639LANGNAME, isolang, sizeof(isolang));
+ GetLocaleInfoA(lcid, LOCALE_SISO3166CTRYNAME, isocrty, sizeof(isocrty));
+ snprintf(iso_lc_messages, sizeof(iso_lc_messages) - 1, "%s_%s", isolang, isocrty);
+ return iso_lc_messages;
+ }
+ return NULL;
+ #else
+ return NULL; /* does nothing. */
+ #endif /* _MSC_VER >= 1400 */
+ }
+ #endif /* WIN32 */
+
*** a/src/include/port/win32.h
--- b/src/include/port/win32.h
***************
*** 291,296 **** extern int pgwin32_is_service(void);
--- 291,301 ----
/* in port/win32error.c */
extern void _dosmaperr(unsigned long);
+ /* in port/win32env.c */
+ extern int pgwin32_putenv(const char *);
+ extern void pgwin32_unsetenv(const char *);
+ #define putenv(x) pgwin32_putenv(x)
+ #define unsetenv(x) pgwin32_unsetenv(x)
/* Things that exist in MingW headers, but need to be added to MSVC */
#ifdef WIN32_ONLY_COMPILER
*** /dev/null
--- b/src/port/win32env.c
***************
*** 0 ****
--- 1,92 ----
+ /*-------------------------------------------------------------------------
+ *
+ * win32env.c
+ * putenv() and unsetenv() for win32, that updates both process
+ * environment and the cached versions in (potentially multiple)
+ * MSVCRT.
+ *
+ * Portions Copyright (c) 1996-2009, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * $PostgreSQL$
+ *
+ *-------------------------------------------------------------------------
+ */
+
+ #include "c.h"
+
+ int pgwin32_putenv(const char *envval)
+ {
+ char *envcpy;
+ char *cp;
+
+ /*
+ * Each version of MSVCRT has its own _putenv() call in the runtime
+ * library.
+ *
+ * If we're in VC 7.0 or later (means != mingw), update in
+ * the 6.0 MSVCRT.DLL environment as well, to work with third party
+ * libraries linked against it (such as gnuwin32 libraries).
+ */
+ #if defined(_MSC_VER) && (_MSC_VER >= 1300)
+ typedef int (_cdecl *PUTENVPROC)(const char *);
+ HMODULE hmodule;
+ static PUTENVPROC putenvFunc = NULL;
+ int ret;
+
+ if (putenvFunc == NULL)
+ {
+ hmodule = GetModuleHandle("msvcrt");
+ if (hmodule == NULL)
+ return 1;
+ putenvFunc = (PUTENVPROC)GetProcAddress(hmodule, "_putenv");
+ if (putenvFunc == NULL)
+ return 1;
+ }
+ ret = putenvFunc(envval);
+ if (ret != 0)
+ return ret;
+ #endif /* _MSC_VER >= 1300 */
+
+
+ /*
+ * Update the process environment - to make modifications visible
+ * to child processes.
+ *
+ * Need a copy of the string so we can modify it.
+ */
+ envcpy = strdup(envval);
+ cp = strchr(envcpy, '=');
+ if (cp == NULL)
+ return -1;
+ *cp = '\0';
+ cp++;
+ if (strlen(cp) == 0)
+ cp = NULL;
+ if (!SetEnvironmentVariable(envcpy, cp))
+ {
+ free(envcpy);
+ return -1;
+ }
+ free(envcpy);
+
+ /* Finally, update our "own" cache */
+ return _putenv(envval);
+ }
+
+ void
+ pgwin32_unsetenv(const char *name)
+ {
+ char *envbuf;
+
+ envbuf = (char *) malloc(strlen(name)+2);
+ if (!envbuf)
+ return;
+
+ sprintf(envbuf, "%s=", name);
+ pgwin32_putenv(envbuf);
+ free(envbuf);
+ }
+
*** a/src/tools/msvc/Mkvcbuild.pm
--- b/src/tools/msvc/Mkvcbuild.pm
***************
*** 44,53 **** sub mkvcbuild
our @pgportfiles = qw(
chklocale.c crypt.c fseeko.c getrusage.c inet_aton.c random.c srandom.c
! unsetenv.c getaddrinfo.c gettimeofday.c kill.c open.c rand.c
snprintf.c strlcat.c strlcpy.c copydir.c dirmod.c exec.c noblock.c path.c pipe.c
pgsleep.c pgstrcasecmp.c qsort.c qsort_arg.c sprompt.c thread.c
! getopt.c getopt_long.c dirent.c rint.c win32error.c);
$libpgport = $solution->AddProject('libpgport','lib','misc');
$libpgport->AddDefine('FRONTEND');
--- 44,53 ----
our @pgportfiles = qw(
chklocale.c crypt.c fseeko.c getrusage.c inet_aton.c random.c srandom.c
! getaddrinfo.c gettimeofday.c kill.c open.c rand.c
snprintf.c strlcat.c strlcpy.c copydir.c dirmod.c exec.c noblock.c path.c pipe.c
pgsleep.c pgstrcasecmp.c qsort.c qsort_arg.c sprompt.c thread.c
! getopt.c getopt_long.c dirent.c rint.c win32env.c win32error.c);
$libpgport = $solution->AddProject('libpgport','lib','misc');
$libpgport->AddDefine('FRONTEND');
Magnus Hagander wrote: > Hiroshi Inoue wrote: >> Hiroshi Inoue wrote: >>> Magnus Hagander wrote: >>>> Do you want to send an updated patch for it, or do you want me to look >>>> at it? >>> I would send a new patch to which I added a simple ISO style check for >>> locale names. >> Attached is a new patch. >> I added a simple ISO style locale name check. >> Avoided codings like *NULL == somthing*. >> It also includes the changes to mbutils.c and elog.c which fix >> recently introduced bug by the domain name change from "postgres" >> to "postgres-8.4". > > Attached is a further updated version of this patch. Changes include: Thanks. > * Actually avoid NULL==something style coding everywhere (I think) > * Avoid coding like "if (ret = putenvFunc(envval), 0 != ret)" > * Per discussion, remove pg_locale.c specific coding, put it in > src/port, and do a redefine so we *always* use these new functions > * Some further minor cleanups > * Removed the change to mbutils.c and elog.c - those are a separate > issue, will deal with those as a separate patch. > > There still needs to be some error checking added in IsoLocaleName(), > but this is a start. > > Can someone please test this? :-) OK I would check it tonight. Please note the patch relies on the proper gettext module. The header, lib and dlls are placed on Saito-san's site http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/ . Also note the current source tries to find the message catalog postgres-8.4.mo not postgres.mo. The attached is an example patch to change the name of mo files at installation time. I'm not good at perl and there may be cleaner way. regards, Hiroshi Inoue *** Install_orig.pm Wed Sep 17 13:31:08 2008 --- Install.pm Sat Jan 10 21:15:59 2009 *************** *** 460,465 **** --- 460,478 ---- print "Installing NLS files..."; EnsureDirectories($target, "share/locale"); + + my $majorver; + open(F, "src/include/pg_config.h") || confess "Could not open file pg_config.h\n"; + while (<F>) + { + if (/^#define\s+PG_MAJORVERSION\s+"([^"]+)"/) + { + $majorver = $1; + } + } + print "majorver=$majorver\n"; + close(F); + my @flist; File::Find::find({wanted => sub { /^nls\.mk\z/s && *************** *** 481,487 **** EnsureDirectories($target, "share/locale/$lang", "share/locale/$lang/LC_MESSAGES"); system( ! "\"$nlspath\\bin\\msgfmt\" -o \"$target\\share\\locale\\$lang\\LC_MESSAGES\\$prgm.mo\" $_" ) && croak("Could not run msgfmt on $dir\\$_"); print "."; --- 494,500 ---- EnsureDirectories($target, "share/locale/$lang", "share/locale/$lang/LC_MESSAGES"); system( ! "\"$nlspath\\bin\\msgfmt\" -o \"$target\\share\\locale\\$lang\\LC_MESSAGES\\$prgm-$majorver.mo\" $_" ) && croak("Could not run msgfmt on $dir\\$_"); print ".";
Hiroshi Inoue wrote: > Magnus Hagander wrote: >> There still needs to be some error checking added in IsoLocaleName(), >> but this is a start. >> >> Can someone please test this? :-) > > OK I would check it tonight. Thanks. > Please note the patch relies on the proper gettext module. > The header, lib and dlls are placed on Saito-san's site > http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/libintl_check/ Aha, that I missed. > Also note the current source tries to find the message catalog > postgres-8.4.mo not postgres.mo. > The attached is an example patch to change the name of mo files > at installation time. I'm not good at perl and there may be > cleaner way. I've rewritten this a little bit, and will apply separately (when I have a little more bandwidth to work with) since I assume this is needed even without the other patch (for the non-changeable way we had before) //Magnus
Magnus Hagander wrote:
> Hiroshi Inoue wrote:
>> Magnus Hagander wrote:
>>> There still needs to be some error checking added in IsoLocaleName(),
>>> but this is a start.
>>>
>>> Can someone please test this? :-)
>> OK I would check it tonight.
>
> Thanks.
OK seems to works here.
The attached is a test case using lc_messages C, fr, de, es and sv.
regards,
Hiroshi Inoue
SET
lc_messages
-------------
c
(1 row)
ERROR: syntax error at or near "1"
LINE 1: 1;
^
SET
lc_messages
-------------
fr
(1 row)
ERREUR: erreur de syntaxe sur ou près de « 1 »
LINE 1: 1;
^
SET
lc_messages
-------------
de
(1 row)
FEHLER: Fehler »Syntaxfehler« bei »1«
LINE 1: 1;
^
SET
lc_messages
-------------
es
(1 row)
ERROR: error de sintaxis en o cerca de «1»
LINE 1: 1;
^
SET
lc_messages
-------------
sv
(1 row)
FEL: syntaxfel vid eller nära "1"
LINE 1: 1;
^
Hiroshi Inoue wrote: > Magnus Hagander wrote: >> Hiroshi Inoue wrote: >>> Magnus Hagander wrote: >>>> There still needs to be some error checking added in IsoLocaleName(), >>>> but this is a start. >>>> >>>> Can someone please test this? :-) >>> OK I would check it tonight. >> >> Thanks. > > OK seems to works here. > The attached is a test case using lc_messages C, fr, de, es and sv. I have applied this version of the patch with some more error checking added to the ISO locale function. Along with the separate commit for the .mo file naming, this should hopefully take care of this issue? //Magnus