Re: [pgsql-zh-general] Chinese in Postgres - Mailing list pgsql-ru-general

From Huang Bambo
Subject Re: [pgsql-zh-general] Chinese in Postgres
Date
Msg-id CAPTvEdoxUJ+4umZ14B_kQL0VYfRBATgTjtzcKxTSfVx9QuuyVQ@mail.gmail.com
Whole thread Raw
In response to Chinese in Postgres  ("ciifrancesco@tiscali.it" <ciifrancesco@tiscali.it>)
List pgsql-ru-general
Hi, Francesco,
As I mentioned that you said "If I insert the data using a C++ program I have empty squares", I guess you forget to convert you string to UTF-8 before insert.

Bambo


On Fri, Aug 16, 2013 at 7:25 PM, ciifrancesco@tiscali.it <ciifrancesco@tiscali.it> wrote:
Hello all,
before writing this message, I wrote about this in other mailing lists without solving my problem.
Maybe some of you can help me.

I have problems with a DB in postgres, when i try to insert Chinese strings in UTF-8 format.
If I insert the data using a C++ program I have empty squares, in this format: ��� (3 empty squares for each chinese ideogram as that is the length in UTF-8)
If the string contains chinese mixed with ASCII, the ASCII is OK but the Chinese is broken:
漢語1-3漢語  --> ������1-3������

All the data is read from a binary file. It seems it's read correctly, but something happens when the query is executed.
(If the text is in a different language that uses only 2 bytes for each letter, I will see only 2 empty squares per character, ex. hebrew, but this is not good anyway...)

Strange things:
1. if i insert the record doing a query from command line (putty), the chinese text is OK. This problem is only when i insert by the C++ program.
2. I checked the C++ functions involved by creating unitary tests; if i run some other tests (on another virtual machine) the text is not damaged.
These strange things are confusing me, but maybe they will be useful informations for somebody who had the same problem.

The DB is set for UTF-8
     Name     | Owner | Encoding |   Collate   |    Ctype    | Access privileges
--------------+-------+----------+-------------+-------------+------------------
postgres     | pgsql | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
MyDB         | pgsql | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
template0    | pgsql | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
template1    | pgsql | UTF8     | en_US.UTF-8 | en_US.UTF-8 |


Previously I also tried with:

     Name     | Owner | Encoding |   Collate   |    Ctype    | Access privileges
--------------+-------+----------+-------------+-------------+------------------
postgres     | pgsql | UTF8     | C           | C           |
MyDB         | pgsql | UTF8     | C           | C           |

...

But the problem was the same.
I know that you would like to see the code, but it's too long (anyway if you want i can try to write some lines of code, like connection to Db and so on). I don't know if there is some log create by postgres when inserting damaged data, sould be useful.

For now, in order to save your time my question is: did anybody of you have the same problem?
(and how did you solve it?)

Thanks,
Francesco


Invita i tuoi amici e Tiscali ti premia! Il consiglio di un amico vale più di uno spot in TV. Per ogni nuovo abbonato 30 € di premio per te e per lui! Un amico al mese e parli e navighi sempre gratis: http://freelosophy.tiscali.it/


pgsql-ru-general by date:

Previous
From: "ciifrancesco@tiscali.it"
Date:
Subject: R: 回复: [pgsql-zh-general] R: Re: Chinese in Postgres
Next
From: Peter Geoghegan
Date:
Subject: Re: [HACKERS] Chinese in Postgres