Re: character conversion problem about UTF-8-->SHIFT_JIS_2004 - Mailing list pgsql-general

Hi.

Oh ok,
About the data in a database.
From the pre- report, you are operating by UTF-8. Therfore,
you should take it out by UTF-8 first. It is reload to new 8.3 of UTF-8.
Then, conversion does not occur there.

To the next.
when SJIS is required by the client , some character codes may not have
a conversion table. Then, It will catch with another problem.
It is necessary to analyze it in detail. However, A database can be operated
normally.

Regards,
Hiroshi Saito

----- Original Message -----
From: "bh yuan" <bhyuan@gmail.com>


>I think i can export olddb(7.4) by encoding SJIS,
> but can not import it into new db(8.3) for some unregular sjis character.
>
> And
> I create the database by encoding UTF-8 .
> ---
> createdb -E UNICODE db
> ---
> I think it is the same to set PGCLIENTENCODING=UTF-8
> .
> use set client_encoding=''SJIS be the same to export PGCLIENTENCODING=SJIS.
>
> But can not export the data to csv file corecttly without errror,
> just because the character which is not SJIS encoding.
>
> The main problem is that , I want to ignore the none SJIS character,
> but I do not know how todo it ... :(
> Such as iconv function in php
> -----------------------------
> If you append the string //TRANSLIT to out_charset transliteration is
> activated. This means that when a character can't be represented in
> the target charset, it can be approximated through one or several
> similarly looking characters. If you append the string //IGNORE,
> characters that cannot be represented in the target charset are
> silently discarded. Otherwise, str is cut from the first illegal
> character.
> -----------------------------
>
> Of course it is not the best way ,
> but I just want to resolve this problem immediately
>
>
> Thanks
>
> 2008/2/8, Hiroshi Saito <z-saito@guitar.ocn.ne.jp>:
>> Oops again..
>>
>> or
>> --
>> set PGCLIENTENCODING=SJIS
>> pg_dump
>> or
>> copy of psql's.
>> --
>> However, an environmental setup is as follows again.
>>
>> export PGCLIENTENCODING=SJIS
>> :-)
>>
>> Regards,
>> Hiroshi Saito
>>
>> ----- Original Message -----
>> From: "Hiroshi Saito" <z-saito@guitar.ocn.ne.jp>
>>
>>
>> > Hi.
>> >
>> > ----- Original Message -----
>> > From: "bh yuan" <bhyuan@gmail.com>
>> >
>> >
>> >> Thanks for your replay.
>> >>
>> >> The  "0xe9ab99" is not SJIS nor SHIFT_JIS_2004.
>> >
>> > Ahh Ok, you already understood.  :-)
>> >
>> >> But I shoud export data with not regular SJIS character
>> >> from old database(7.4.3) to new database(8.3),
>> >> and use the old programe which export data as SJIS encoding .csv file.
>> >> Can I modify conf file to ignore the error?
>> >> or check inigore character from the database and convert it to regular
>> >> SJIS encoding character with some tools ?
>> >
>> > Although it is unknown in whether the inside of .csv file which you acquired
>> > is SJIS...However, as one method,
>> > If you were operating by SERVER_ENCODING of UTF-8, it will be able to
>> > bring without conversion. Then, an environment variable will be helpful.
>> >
>> > --
>> > set PGCLIENTENCODING=UTF-8
>> > pg_dump
>> > --
>> >
>> > Regards,
>> > Hiroshi Saito
>> >
>> >>
>> >> And
>> >>> >> to export data as .csv file,
>> >>> >> I use  set client_encoding='SJIS' at client.
>> >>> >
>> >>> > No, you should use UTF-8 of default.
>> >> means export a UTF-8 encoding csv file?
>> >>
>> >> Thanks
>> >>
>> >> 2008/2/8, Hiroshi Saito <z-saito@guitar.ocn.ne.jp>:
>> >>> Ooops, shortage of information..sorry.
>> >>> Please see,
>> >>> http://winpg.jp/~saito/pg83/HASHIGODATA/
>> >>>
>> >>> > Hi.
>> >>> >
>> >>> > ----- Original Message -----
>> >>> > From: "bh yuan" <bhyuan@gmail.com>
>> >>> >
>> >>> >> hi
>> >>> >>
>> >>> >> I used Postgresql7.4.3 with php for more than 3years.
>> >>> >> Now I want to change my database to Postgresql8.3.
>> >>> >> But I occur such problem
>> >>> >> ----------------------------------------------------------
>> >>> >> ERROR: character 0xe9ab99 of encoding "UTF8" has no equivalent in "SJIS"
>> >>> >> ERROR: character 0xe9ab99 of encoding "UTF8" has no equivalent in
>> >>> >> "SHIFT_JIS_2004"
>> >>> >> ----------------------------------------------------------
>> >>> >> The database was encoded by UTF-8,
>> >>> >
>> >>> > It SERVER_ENCODING=UTF-8 is Ok.
>> >>> >
>> >>> >> to export data as .csv file,
>> >>> >> I use  set client_encoding='SJIS' at client.
>> >>> >
>> >>> > No, you should use UTF-8 of default.
>> >>> >
>> >>> >> When I use Postgresql7.4.3,no problem occur,
>> >>> >
>> >>> > It seems that it has loose check....
>> >>> >
>> >>> >> but after I chaged to Postgresql8.3 ,the error was occured.
>> >>> >>
>> >>> >> Can I ignore the error message ?
>> >>> >> or any othe method to solve this problem.
>> >>> >
>> >>> > "0xe9ab99" which you use is famous UNICODE.
>> >>> > Then, An error is right. (not SJIS)
>> >>> >
>> >>> > Regards,
>> >>> > Hiroshi Saito
>> >>>
>> >>
>> >> ---------------------------(end of broadcast)---------------------------
>> >> TIP 3: Have you checked our extensive FAQ?
>> >>
>> >>               http://www.postgresql.org/docs/faq
>> >
>> > ---------------------------(end of broadcast)---------------------------
>> > TIP 1: if posting/reading through Usenet, please send an appropriate
>> >       subscribe-nomail command to majordomo@postgresql.org so that your
>> >       message can get through to the mailing list cleanly
>>