Thread: ODBC & character encoding
Hi,
I am trying to run a query from .Net C# using ODBC on a Postgres 7.1 Database.
The database encoding is EUC_JP.
And as I use .Net the default encoding is UTF8.
My problem is that the 7.1 version does not support to set the client encoding to UTF8 :
Here is the ODBC error log:
<code>
[ Client encoding = 'EUC_JP' (code = 1) ]
conn=105462528, query='set client_encoding to 'UTF8''
ERROR from backend during send_query: 'ERROR: Client encoding UTF8 is not supported'
conn=105462528, query='set client_encoding to 'UTF8''
ERROR from backend during send_query: 'ERROR: Client encoding UTF8 is not supported'
</code>
So when I insert Data, the japanese characters don't appear properly....
The database is running for a production server and I cannot stop the service at all... Not to mention upgrading...
So my question is :
- Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?
and how ?
I hope you understand my problem and my question...
Any help would be extremely appreciated,
Benjamin
Hi,
I havn't dne much with unicode on a PG database yet, so this is just a guess (albeit an educated one based on what i read in the docs)
Maybe you need to use "set client_encoding to 'UNICODE'" as there didn't appear to be a UTF8 encoding as such.
refer to section 20.2.1 of the 7.4 manual: http://www.postgresql.org/docs/7.4/interactive/multibyte.html - don't think this has changed since 7.1, but you should check it.
regards
Iain
----- Original Message -----From: Layet BenjaminSent: Monday, February 07, 2005 4:04 PMSubject: [ODBC] ODBC & character encodingHi,I am trying to run a query from .Net C# using ODBC on a Postgres 7.1 Database.The database encoding is EUC_JP.And as I use .Net the default encoding is UTF8.My problem is that the 7.1 version does not support to set the client encoding to UTF8 :Here is the ODBC error log:<code>[ Client encoding = 'EUC_JP' (code = 1) ]
conn=105462528, query='set client_encoding to 'UTF8''
ERROR from backend during send_query: 'ERROR: Client encoding UTF8 is not supported'</code>So when I insert Data, the japanese characters don't appear properly....The database is running for a production server and I cannot stop the service at all... Not to mention upgrading...So my question is :- Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?and how ?I hope you understand my problem and my question...Any help would be extremely appreciated,Benjamin
Hi,
Thank you Iain for your quick reply.
But unfortunately this did not solve the problem, 7.1 does not support this conversion neither yet. (7.3 and 7.4 do):
SET client_encoding='UNICODE';
ERROR: Conversion between UNICODE and EUC_JP is not supported
ERROR: Conversion between UNICODE and EUC_JP is not supported
In theory, I have 3 possible places to convert UTF8 to EUC_JP:
- 1 inside postgres
- 2 inside ODBC
- 3 inside C#
1
C# ODBC Postgres 7.1
Utf8 l =============> l l =====>l UTF8->EUC_JP
(possible->Unicode) l UTF8 l l UTF8 l
______________________l (or Unicode) l________________l l_______________
Utf8 l =============> l l =====>l UTF8->EUC_JP
(possible->Unicode) l UTF8 l l UTF8 l
______________________l (or Unicode) l________________l l_______________
2
C# ODBC Postgres 7.1
Utf8 l =============> l UTF8->EUC_JP l =====>l
(possible->Unicode) l UTF8 l l EUC_JPl
______________________l (or Unicode) l________________l l_______________
Utf8 l =============> l UTF8->EUC_JP l =====>l
(possible->Unicode) l UTF8 l l EUC_JPl
______________________l (or Unicode) l________________l l_______________
3
C# ODBC Postgres 7.1
UTF8->EUC_JP l =============> l l =====>l
(possible->Unicode) l EUC_JP l l EUC_JPl
______________________l l________________l l_______________
Number 1 seem to be the normal solution, but it does not work with 7.1.
I would like to know if number 2 or 3 are possible:
so,
- Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?
or
- Can the ODBC driver take EUC_JP string as input ? (then I would 'just' have to figure out how to send EUC_JP stings from C#...)
Help extremely needed...
Thank you !
Benjamin
Hi Benjamin,
I don't have direct experience of your exact situation, but I use a 7.1 database via ODBC. The difference is that the client is not unicode, it's plain old windows shift-JIS and that works fine iff you use the unicode support odbc driver. When you install the postgres ODBC driver it actually contains 3 drivers, which one are you using?
> Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?
I'm not sure...
> Can the ODBC driver take EUC_JP string as input ? (then I would 'just' have to figure out how to send EUC_JP stings from C#...)
I dunno about that either, but it certainly accepts shift-JIS which is the native encoding for windows.
So far I've never had to pay much attention to client encoding (with the notable exception of getting input data from a web application) as the clients and drivers seem to work everything out by themselves.
Regards
Iain
----- Original Message -----From: Layet BenjaminSent: Monday, February 07, 2005 5:06 PMSubject: Re: [ODBC] ODBC & character encodingHi,Thank you Iain for your quick reply.But unfortunately this did not solve the problem, 7.1 does not support this conversion neither yet. (7.3 and 7.4 do):SET client_encoding='UNICODE';
ERROR: Conversion between UNICODE and EUC_JP is not supportedIn theory, I have 3 possible places to convert UTF8 to EUC_JP:- 1 inside postgres- 2 inside ODBC- 3 inside C#1C# ODBC Postgres 7.1
Utf8 l =============> l l =====>l UTF8->EUC_JP
(possible->Unicode) l UTF8 l l UTF8 l
______________________l (or Unicode) l________________l l_______________2C# ODBC Postgres 7.1
Utf8 l =============> l UTF8->EUC_JP l =====>l
(possible->Unicode) l UTF8 l l EUC_JPl
______________________l (or Unicode) l________________l l_______________3
C# ODBC Postgres 7.1
UTF8->EUC_JP l =============> l l =====>l
(possible->Unicode) l EUC_JP l l EUC_JPl
______________________l l________________l l_______________Number 1 seem to be the normal solution, but it does not work with 7.1.I would like to know if number 2 or 3 are possible:so,- Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?or- Can the ODBC driver take EUC_JP string as input ? (then I would 'just' have to figure out how to send EUC_JP stings from C#...)Help extremely needed...Thank you !Benjamin
Hi Iain,
Thank you again for your reply.
It's good to know that things works for people almost in my situation :-).
I am not surprise that there is no problem with Shift-Jis, since "SET client_encoding= 'SJIS';" works fine on a EUC_JP database.
But "SET client_encoding= 'UNICODE';" does not work with 7.1.... and I feel really stuck because of that...
>When you install the postgres ODBC driver it actually contains 3 drivers, which one are you using?
I tried the PostgreSQL 8.00. and PostgreSQL Unicode beta but both of them try to do "SET client_encoding= 'UNICODE'"...
In my case, I really need to convert the encoding before sending it to Postgres... but I don't know if it is possible.
>So far I've never had to pay much attention to client encoding... Lucky man :-)
Cheers,
Ben
----- Original Message -----From: IainSent: Monday, February 07, 2005 5:31 PMSubject: Re: [ODBC] ODBC & character encodingHi Benjamin,I don't have direct experience of your exact situation, but I use a 7.1 database via ODBC. The difference is that the client is not unicode, it's plain old windows shift-JIS and that works fine iff you use the unicode support odbc driver. When you install the postgres ODBC driver it actually contains 3 drivers, which one are you using?> Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?I'm not sure...> Can the ODBC driver take EUC_JP string as input ? (then I would 'just' have to figure out how to send EUC_JP stings from C#...)I dunno about that either, but it certainly accepts shift-JIS which is the native encoding for windows.So far I've never had to pay much attention to client encoding (with the notable exception of getting input data from a web application) as the clients and drivers seem to work everything out by themselves.RegardsIain----- Original Message -----From: Layet BenjaminSent: Monday, February 07, 2005 5:06 PMSubject: Re: [ODBC] ODBC & character encodingHi,Thank you Iain for your quick reply.But unfortunately this did not solve the problem, 7.1 does not support this conversion neither yet. (7.3 and 7.4 do):SET client_encoding='UNICODE';
ERROR: Conversion between UNICODE and EUC_JP is not supportedIn theory, I have 3 possible places to convert UTF8 to EUC_JP:- 1 inside postgres- 2 inside ODBC- 3 inside C#1C# ODBC Postgres 7.1
Utf8 l =============> l l =====>l UTF8->EUC_JP
(possible->Unicode) l UTF8 l l UTF8 l
______________________l (or Unicode) l________________l l_______________2C# ODBC Postgres 7.1
Utf8 l =============> l UTF8->EUC_JP l =====>l
(possible->Unicode) l UTF8 l l EUC_JPl
______________________l (or Unicode) l________________l l_______________3
C# ODBC Postgres 7.1
UTF8->EUC_JP l =============> l l =====>l
(possible->Unicode) l EUC_JP l l EUC_JPl
______________________l l________________l l_______________Number 1 seem to be the normal solution, but it does not work with 7.1.I would like to know if number 2 or 3 are possible:so,- Can the ODBC driver convert the query string from UTF8 to EUC_JP before sending it to the database server ?or- Can the ODBC driver take EUC_JP string as input ? (then I would 'just' have to figure out how to send EUC_JP stings from C#...)Help extremely needed...Thank you !Benjamin