Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 - Mailing list pgsql-hackers

That is pretty much where we are ;-)
I think we're fine for 8.0.x with this, because if you actually need
UTF-8 (and can live with sorting broken, no upper/lower etc), you can do
it using a manual initdb.

For 8.1, I think the ICU approach looks a lot more promising than trying
to do "on the fly conversion to UTF-16 and back". Especially if there is
profit in having ICU for other platforms as well, since we would do
without win32 specific code for that (I seem to recall there being
discussions about other platforms needing it as well - and the guy who
did it didn't do it for win32, so there is at least some..)

I was planning to test the ICU patch on win32 to see that it works at
all, but I haven't had the time to do that just yet.

//Magnus


>Where are we on this?  As far as I can tell, we never disabled UTF8 on
>Win32 in our code.  The only thing we did do was to disable UTF8 in
>pginstaller.  See this FAQ item:
>
>
>http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6
>
>Is the current setup OK?  Should we allow UTF8 on Win32 for languages
>that can use C locale, like Asian languages?
>
>---------------------------------------------------------------
>------------
>
>Tatsuo Ishii wrote:
>> I do understand the problem, but don't undertstand the decision you
>> guys made. The fact that UPPER/LOWER and some other
>functions does not
>> work in win32 is surely a problem for some languages, but not a
>> problem for otheres. For example, Japanese (and probably Chinese and
>> Korean) does not have a concept upper/lower. So the fact UPPER/LOWER
>> does not work with UTF-8/win32 is not problem for Japanese (and for
>> some other languages). Just using C locale with UTF-8 is enough in
>> this case.
>>
>> In summary, I think you guys are going to overkill the multibyte
>> support functionality on UTF-8/win32 because of the fact that some
>> langauges do not work.
>>
>> Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so
>on as well.
>>
>> I strongly object the policy to try to unconditionaly disable UTF-8
>> support on win32.
>> --
>> Tatsuo Ishii
>>
>> From: "Magnus Hagander" <mha@sollentuna.net>
>> Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32
>> Date: Sat, 1 Jan 2005 14:48:04 +0100
>> Message-ID:
><6BCB9D8A16AC4241919521715F4D8BCE4764A4@algol.sollentuna.se>
>>
>> > UNICODE/UTF-8 does not work on the win32 server. The reason is that
>> > strcoll() and friends don't work with it. To support it on
>win32, it
>> > needs to be converted to UTF16 and use the wide-character
>versions of
>> > the fucntion. Which we do not do.
>> > (See
>> >
>http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036.php
>> > and
>> >
>http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg0
>0106.php)
>> >
>> >
>> > I don't *think* we need to disable ito n the client.
>AFAIK, the client
>> > interfaces don't use any of these functions, and I've seen
>reports of
>> > people using that long before we had a native win32 server.
>> >
>> >
>> > //Magnus
>> >
>> >
>> > >-----Original Message-----
>> > >From: Tatsuo Ishii [mailto:t-ishii@sra.co.jp]
>> > >Sent: den 1 januari 2005 01:10
>> > >To: tgl@sss.pgh.pa.us
>> > >Cc: Magnus Hagander; pgsql-hackers-win32@postgresql.org
>> > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32
>> > >
>> > >
>> > >Sorry, but I don't subscribe to pgsql-hackers-win32 list.
>What's the
>> > >problem here?
>> > >--
>> > >Tatsuo Ishii
>> > >
>> > >> "Magnus Hagander" <mha@sollentuna.net> writes:
>> > >> > We know it's broken and won't be fixed for 8.0.
>> > >>
>> > >> > If we just #ifndef WIN32 the definitions in
>> > >utils/mb/encnames.c it won't
>> > >> > be possible to select that encoding, right? Will that have
>> > >any other
>> > >> > unwanted effects (such as breaking client encodings)? If
>> > >not, I suggest
>> > >> > this is done.
>> > >>
>> > >> I believe the subscripts in those arrays have to match
>the encoding
>> > >> enum type, so you can't just ifdef out individual entries.
>> > >>
>> > >> > (Or perhaps something can be done in
>pg_valid_server_encoding?)
>> > >>
>> > >> Making the valid_server_encoding function reject it might work.
>> > >> Tatsuo-san would know for sure.
>> > >>
>> > >> Should we also reject it as a client encoding, or does
>that work OK?
>> > >>
>> > >>             regards, tom lane
>> > >>
>> > >
>> >
>>
>> ---------------------------(end of
>broadcast)---------------------------
>> TIP 1: subscribe and unsubscribe commands go to
>majordomo@postgresql.org
>>
>
>--
>  Bruce Momjian                        |  http://candle.pha.pa.us
>  pgman@candle.pha.pa.us               |  (610) 359-1001
>  +  If your life is a hard drive,     |  13 Roberts Road
>  +  Christ can be your backup.        |  Newtown Square,
>Pennsylvania 19073
>