Thread: Usind database with encoding UNICODE and vovel

Hi anybody,

Encironment: Win2k, german locale, PostgreSQL database with UNICODE encoding, using via ODBC
                    Client encoding UNICODE.

I can read strings containing vovels without problems, But writing strings containing vovels does not work !
Can anybody assist ?

mit freundlichen Grüssen,
Josef Springer
(Geschäftsleitung)

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Josef Springer wrote:
> Hi anybody,
>
> Encironment: Win2k, german locale, PostgreSQL database with UNICODE
> encoding, using via ODBC
>                     Client encoding UNICODE.
>
> I can read strings containing vovels without problems, But writing
> strings containing vovels does not work !
> Can anybody assist ?

I've been posting a patch earlier this year (January) because I had the
same problem on Unicode databases. But this is not the ultimate patch;
it will create data errors with non-unicode databases (e.g. Latin-1). I
wonder if the odbc driver changes the client encoding to unicode, as it
should when connecting to non-ASCII databases.

Regards,
Andreas

Hi Andreas,

do you know wether your patch or this problem is corrected in the
corrint version 8.0.1 ?

Josef Springer

Andreas Pflug wrote:

> Josef Springer wrote:
>
>> Hi anybody,
>>
>> Encironment: Win2k, german locale, PostgreSQL database with UNICODE
>> encoding, using via ODBC
>>                     Client encoding UNICODE.
>>
>> I can read strings containing vovels without problems, But writing
>> strings containing vovels does not work !
>> Can anybody assist ?
>
>
> I've been posting a patch earlier this year (January) because I had
> the same problem on Unicode databases. But this is not the ultimate
> patch; it will create data errors with non-unicode databases (e.g.
> Latin-1). I wonder if the odbc driver changes the client encoding to
> unicode, as it should when connecting to non-ASCII databases.
>
> Regards,
> Andreas
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>               http://archives.postgresql.org

> -----Original Message-----
> From: Andreas Pflug [mailto:pgadmin@pse-consulting.de]
> Sent: 03 August 2005 09:28
> To: Josef Springer
> Cc: pgsql-odbc@postgresql.org; Dave Page
> Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
>
> Josef Springer wrote:
> > Hi anybody,
> >
> > Encironment: Win2k, german locale, PostgreSQL database with UNICODE
> > encoding, using via ODBC
> >                     Client encoding UNICODE.
> >
> > I can read strings containing vovels without problems, But writing
> > strings containing vovels does not work !
> > Can anybody assist ?
>
> I've been posting a patch earlier this year (January) because
> I had the
> same problem on Unicode databases. But this is not the
> ultimate patch;
> it will create data errors with non-unicode databases (e.g.
> Latin-1).

OK, well as you know Andreas, my unicode/locale etc. experience is
pretty much limited to what I've learnt from you and Hiroshi Saito on
pgAdmin, however, I'm on a roll fixing bugs in psqlODBC at the moment,
so let's try to sort this :-)

> I wonder if the odbc driver changes the client encoding to
> unicode, as it
> should when connecting to non-ASCII databases.

No it doesn't but it seems a pretty major thing for Hiroshi (Inoue) to
have missed when he wrote the Unicode support, so I suspect he had some
reason for not doing so at the time

You (or Josef) can do this manually though for testing - in the driver
setup for the DSN, click on DataSource, then on 'Page 2'. Add the
following query to the connect settings box, and OK all the dialogues.

SET client_encoding TO 'unicode'

Then, try it out and let us know what happens!

Regards, Dave.

Server environment:
OS: Win2k (SP 2)
Locale: German
Database version: 8.0.2
Database encoding: UNICODE

Client envorinment:
Same host
Used via ODBC driver (with standard ODBC-driver settings in Win2k ODBC's management, i did not found any documentation about)
First command after connection: SET client_encoding TO 'UNICODE'


Marc Herbert wrote:

On Tue, Aug 02, 2005 at 07:59:23PM +0200, Josef Springer wrote:

Encironment: Win2k, german locale, PostgreSQL database with UNICODE 
encoding, using via ODBC
                   Client encoding UNICODE.

I can read strings containing vovels without problems, But writing 
strings containing vovels does not work !

Do you mean signs with diacritics or just plain ASCII vowels ?

Can anybody assist ?

Probably not until you give more details of what happens.

mit freundlichen Grüssen,
Josef Springer
(Geschäftsleitung)

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Hi Dave,

i have tested the settings in driver setup for the DSN without any effect (i send the same command after connected through the port too).
 
The environment:
Server and client same machineWin2k, german locale
PostgrSQL 8.0.3
Using the installed ODBC driver
German locale selected in the PostgreSQL installation

Using no vovels (umlaut) work fine. But using a vovel, curious effect.
I have tried all server and client variations with sense with this result. All tests are done with a string column containing äöüß

Server database encoding	Client encoding	Result
LATIN	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
LATIN	UNICODE	Reading works bur witing gets an error 'Bad code xxx for  UNICODE'
UNICODE	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	WIN1250	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
SQL_ASCII	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0

In PgAdminIII the column is displayed correct in all server database variations !

We plan to announce our Business-Process-Management-System OfficeTalk supportimg PostgreSQL, but without vovels in german .... What to do ?????

Best Regards,
Josef Springer

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Dave Page wrote:

-----Original Message-----
From: Andreas Pflug [mailto:pgadmin@pse-consulting.de] 
Sent: 03 August 2005 09:28
To: Josef Springer
Cc: pgsql-odbc@postgresql.org; Dave Page
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Josef Springer wrote:

Hi anybody,

Encironment: Win2k, german locale, PostgreSQL database with UNICODE 
encoding, using via ODBC
                    Client encoding UNICODE.

I can read strings containing vovels without problems, But writing 
strings containing vovels does not work !
Can anybody assist ?

I've been posting a patch earlier this year (January) because 
I had the 
same problem on Unicode databases. But this is not the 
ultimate patch; 
it will create data errors with non-unicode databases (e.g. 
Latin-1). 

OK, well as you know Andreas, my unicode/locale etc. experience is
pretty much limited to what I've learnt from you and Hiroshi Saito on
pgAdmin, however, I'm on a roll fixing bugs in psqlODBC at the moment,
so let's try to sort this :-)

I wonder if the odbc driver changes the client encoding to 
unicode, as it 
should when connecting to non-ASCII databases.

No it doesn't but it seems a pretty major thing for Hiroshi (Inoue) to
have missed when he wrote the Unicode support, so I suspect he had some
reason for not doing so at the time

You (or Josef) can do this manually though for testing - in the driver
setup for the DSN, click on DataSource, then on 'Page 2'. Add the
following query to the connect settings box, and OK all the dialogues.

SET client_encoding TO 'unicode'

Then, try it out and let us know what happens!

Regards, Dave.

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 07:04
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Hi Dave,

i have tested the settings in driver setup for the DSN without any effect (i send the same command after connected through the port too).
 
The environment:
Server and client same machineWin2k, german locale
PostgrSQL 8.0.3
Using the installed ODBC driver
German locale selected in the PostgreSQL installation

Using no vovels (umlaut) work fine. But using a vovel, curious effect.
I have tried all server and client variations with sense with this result. All tests are done with a string column containing äöüß

Server database encoding	Client encoding	Result
LATIN	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
LATIN	UNICODE	Reading works bur witing gets an error 'Bad code xxx for  UNICODE'
UNICODE	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	WIN1250	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
SQL_ASCII	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0

In PgAdminIII the column is displayed correct in all server database variations !

We plan to announce our Business-Process-Management-System OfficeTalk supportimg PostgreSQL, but without vovels in german .... What to do ?????

Best Regards,
Josef Springer

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Dave Page wrote:

-----Original Message-----
From: Andreas Pflug [mailto:pgadmin@pse-consulting.de] 
Sent: 03 August 2005 09:28
To: Josef Springer
Cc: pgsql-odbc@postgresql.org; Dave Page
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Josef Springer wrote:

Hi anybody,

Encironment: Win2k, german locale, PostgreSQL database with UNICODE 
encoding, using via ODBC
                    Client encoding UNICODE.

I can read strings containing vovels without problems, But writing 
strings containing vovels does not work !
Can anybody assist ?

I've been posting a patch earlier this year (January) because 
I had the 
same problem on Unicode databases. But this is not the 
ultimate patch; 
it will create data errors with non-unicode databases (e.g. 
Latin-1). 

OK, well as you know Andreas, my unicode/locale etc. experience is
pretty much limited to what I've learnt from you and Hiroshi Saito on
pgAdmin, however, I'm on a roll fixing bugs in psqlODBC at the moment,
so let's try to sort this :-)

I wonder if the odbc driver changes the client encoding to 
unicode, as it 
should when connecting to non-ASCII databases.

No it doesn't but it seems a pretty major thing for Hiroshi (Inoue) to
have missed when he wrote the Unicode support, so I suspect he had some
reason for not doing so at the time

You (or Josef) can do this manually though for testing - in the driver
setup for the DSN, click on DataSource, then on 'Page 2'. Add the
following query to the connect settings box, and OK all the dialogues.

SET client_encoding TO 'unicode'

Then, try it out and let us know what happens!

Regards, Dave.

Hi Dave,

thanks for quick answer.

• I used Win2k with german locale
• Used the ODBC 08.00.0101 (psqlodbc.dll)

My application works well with other databases.
Is a point of  this the reason of my problems ?

What locale settings did you use for database and client ?

Best Regards,
Josef Springer

Dave Page wrote:

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 07:04
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Hi Dave,

i have tested the settings in driver setup for the DSN without any effect (i send the same command after connected through the port too).
 
The environment:
Server and client same machineWin2k, german locale
PostgrSQL 8.0.3
Using the installed ODBC driver
German locale selected in the PostgreSQL installation

Using no vovels (umlaut) work fine. But using a vovel, curious effect.
I have tried all server and client variations with sense with this result. All tests are done with a string column containing äöüß

Server database encoding	Client encoding	Result
LATIN	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
LATIN	UNICODE	Reading works bur witing gets an error 'Bad code xxx for  UNICODE'
UNICODE	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	LATIN	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
UNICODE	WIN1250	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0
SQL_ASCII	UNICODE	Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0

In PgAdminIII the column is displayed correct in all server database variations !

We plan to announce our Business-Process-Management-System OfficeTalk supportimg PostgreSQL, but without vovels in german .... What to do ?????

Best Regards,
Josef Springer

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Dave Page wrote:

-----Original Message-----
From: Andreas Pflug [mailto:pgadmin@pse-consulting.de] 
Sent: 03 August 2005 09:28
To: Josef Springer
Cc: pgsql-odbc@postgresql.org; Dave Page
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Josef Springer wrote:

Hi anybody,

Encironment: Win2k, german locale, PostgreSQL database with UNICODE 
encoding, using via ODBC
                    Client encoding UNICODE.

I can read strings containing vovels without problems, But writing 
strings containing vovels does not work !
Can anybody assist ?

I've been posting a patch earlier this year (January) because 
I had the 
same problem on Unicode databases. But this is not the 
ultimate patch; 
it will create data errors with non-unicode databases (e.g. 
Latin-1). 

OK, well as you know Andreas, my unicode/locale etc. experience is
pretty much limited to what I've learnt from you and Hiroshi Saito on
pgAdmin, however, I'm on a roll fixing bugs in psqlODBC at the moment,
so let's try to sort this :-)

I wonder if the odbc driver changes the client encoding to 
unicode, as it 
should when connecting to non-ASCII databases.

No it doesn't but it seems a pretty major thing for Hiroshi (Inoue) to
have missed when he wrote the Unicode support, so I suspect he had some
reason for not doing so at the time

You (or Josef) can do this manually though for testing - in the driver
setup for the DSN, click on DataSource, then on 'Page 2'. Add the
following query to the connect settings box, and OK all the dialogues.

SET client_encoding TO 'unicode'

Then, try it out and let us know what happens!

Regards, Dave.

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 09:45
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
Importance: Low

Hi Dave,

thanks for quick answer.

• I used Win2k with german locale  

• Used the ODBC 08.00.0101 (psqlodbc.dll)  

 

My application works well with other databases.
Is a point of  this the reason of my problems ?

What locale settings did you use for database and client ? 

 

Hi,<br /><br /> I used:<br /><br /> Debian Sarge with PostgreSQL 7.4.7<br /> My application on Win XP SP2<br />
ODBC-driver:08.01.1003<br /><br /> More or less the same problem with LATIN1 (server) and UNICODE (client)<br /> but
PgAdminIIIdoesn't show <b>äöüß</b> later.<br /><span class="150583209-04082005"></span><br /> Regards<br /><br />
Johann<br/><br /> Josef Springer schrieb:<br /><br /><blockquote cite="mid42F1AFEC.1020906@joops.com" type="cite">
Usingno vovels (umlaut) work fine. But using a vovel, curious effect.<br /> I have tried all server and client
variationswith sense with this result. All tests are done with a string column containing <b>äöüß</b><br /><table
border="1"cellpadding="2" cellspacing="2" width="100%"><tbody><tr><td valign="top">Server database encoding<br
/></td><tdvalign="top">Client encoding<br /></td><td valign="top">Result<br /></td></tr><tr><td valign="top">LATIN<br
/></td><tdvalign="top">LATIN<br /></td><td valign="top">Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0<br
/></td></tr><tr><tdvalign="top">LATIN<br /></td><td valign="top">UNICODE</td><td valign="top">Reading works bur witing
getsan error 'Bad code xxx for  UNICODE'<br /></td></tr><tr><td valign="top">UNICODE<br /></td><td
valign="top">UNICODE<br/></td><td valign="top">Writing works. Reading gets the bytearray 63.63,0,1,0,2,47,0<br
/></td></tr><tr><tdvalign="top">UNICODE<br /></td><td valign="top">LATIN<br /></td><td valign="top">Writing works.
Readinggets the bytearray 63.63,0,1,0,2,47,0<br /></td></tr><tr><td valign="top">UNICODE<br /></td><td
valign="top">WIN1250<br/></td><td valign="top">Writing works. Reading gets the bytearray
63.63,0,1,0,2,47,0</td></tr><tr><tdvalign="top">SQL_ASCII<br /></td><td valign="top">UNICODE<br /></td><td
valign="top">Writingworks. Reading gets the bytearray 63.63,0,1,0,2,47,0<br /></td></tr></tbody></table><br
/></blockquote><b>InPgAdminIII the column is displayed correct in all server database variations !</b><br /><br /> 

Hi, I've got the same problem with some spanish chars (ñ,á,é,í) on
driver 8.00.03; with msexcel works fine, but with visualfoxpro don't.
Driver 7.03.02 works fine.

Alejandro.

2005/8/2, Josef Springer <Josef.Springer@joops.com>:
>  Hi anybody,
>
>  Encironment: Win2k, german locale, PostgreSQL database with UNICODE
> encoding, using via ODBC
>                      Client encoding UNICODE.
>
>  I can read strings containing vovels without problems, But writing strings
> containing vovels does not work !
>  Can anybody assist ?
>
> --
>
>
> mit freundlichen Grüssen,
>  Josef Springer
>  (Geschäftsleitung)
>
>   -- the software company --
>
> Orlando-di-Lasso Str. 2
>  D-85640 Putzbrunn
>  Tel. ++49(0)89 600 6920
>  Fax ++49(0)89 600 69220
>  mailto Josef.Springer@joops.com
>  Website http://www.joops.com

> -----Original Message-----
> From: pgsql-odbc-owner@postgresql.org
> [mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of
> Alejandro D. Burne
> Sent: 04 August 2005 11:44
> To: pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
>
> Hi, I've got the same problem with some spanish chars (ñ,á,é,í) on
> driver 8.00.03; with msexcel works fine, but with visualfoxpro don't.
> Driver 7.03.02 works fine.

Do you mean you are using 08.01.0003 or 08.00.0103 ?

What version of Windows/MDAC? What database encoding?

Regards, Dave

He Dave,

as other wrote, there should be an encoding problem with Windows 2000.

My Environment:
Windows 2000 internal version: 5.0.2195; SP 2
Germal locale
PostgreSQL: 8.0.3
ODBC driver: 8.00.01.01 (5. March 2005)
No MDAC

Setting the encoding in the odbc driver (DNS) has no effect.

I made some tests:

Database encoding	Client encoding	Wrinting column (äöüß)	Reading column
UNICODE	LATIN1	OK	corrupted in app
UNICODE	UNICODE	corrupted in database (writing ß -> error character is a bad UNICODE)	OK

Written data was verified with PgAdminIII

Best Regards,
Josef Springer

Dave Page wrote:

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 09:45
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
Importance: Low

Hi Dave,

thanks for quick answer.

• I used Win2k with german locale   

• Used the ODBC 08.00.0101 (psqlodbc.dll)   

 

My application works well with other databases.
Is a point of  this the reason of my problems ?

What locale settings did you use for database and client ?  

 

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 13:35
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

He Dave,

as other wrote, there should be an encoding problem with Windows 2000.

My Environment:
Windows 2000 internal version: 5.0.2195; SP 2
Germal locale
PostgreSQL: 8.0.3
ODBC driver: 8.00.01.01 (5. March 2005)  

 

 
No MDAC
 

Hi Dave,

i have made some more tests:

Database and client encoding both UNICODE.
Writing of sting columns work if the content is converted explicitly to UTF8 before sending.
Reading string columns gets the content without encoding. (one byte for each character).
Thats the behaviour on Win2k.

Best Regards,
Josef Springer

It seems that writing


Dave Page wrote:

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 09:45
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
Importance: Low

Hi Dave,

thanks for quick answer.

• I used Win2k with german locale   

• Used the ODBC 08.00.0101 (psqlodbc.dll)   

 

My application works well with other databases.
Is a point of  this the reason of my problems ?

What locale settings did you use for database and client ?  

 

Dave, thanks 4 your time;
* I've installed MDAC 2.8 with same results.

* If I don't set locale on the client it doesn't work, pgadmin either
(show wrong encoding data)

After connect trough odbc:
SHOW CLIENT_ENCODING;
return -> UTF8

The same command after connect to the LATIN10 db on pgadmin;
SHOW CLIENT_ENCODING;
return -> UNICODE

Try create a LATIN10 db and connect to it to return client encoding.

* About MsQuery I was speaking about visual querys design time, not to
return millons of records.
It works fine with other db, because it find the fields to make the
relationship between tables, right now I'm migrating an app from mysql
to postgres, the same querys, same data on msquery with mysql (myodbc)
works fine; with psqlodbc hangup trying to relate tables (on visual
design, without return a single record), If I type the query it works,
but don't expect final users write a sql sentence =)

* What about the 2 chars used to store strange chars?

Thanks again and again.
Alejandro

2005/8/4, Dave Page <dpage@vale-housing.co.uk>:
>
>
> > -----Original Message-----
> > From: Alejandro D. Burne [mailto:alejandro.dburne@gmail.com]
> > Sent: 04 August 2005 14:13
> > To: Dave Page
> > Subject: Re: Usind database with encoding UNICODE and vovel
> >
> > Ok, XP/MDAC 2.5
>
> 2.5 is quite old now - the current is 2.8 sp1 iirc
>
> > I'll be testing now with 08.01.0003; my first word is works, "but"
> > this strange chars use 2 chars internally in the string
> > SET CLIENT_ENCODING = 'LATIN10';
> > INSERT INTO prueba VALUES ('ÑAÑA');
> > SELECT * FROM prueba;
> > returns 'ÑAÑA' <- WORKS OK
> >
> > just for test..
> > SET CLIENT_ENCODING = 'UNICODE';
> > SELECT * FROM prueba;
> > returns 'A?AA?A' <- 6 chars
>
> In all my testing, I found no need to execute SET CLIENT_ENCODING at all. Please try it without.
>
> > Another tip =)
> > MsExcel returns wrong data.
> > The problem seems to be the client don't select the right encoding for
> > the DB, in this case LATIN10 and I don't know the way (if I can) to do
> > this.
> >
> > The last one... in MsExcel (I guess MsQuery) when I try to relate 2
> > tables don't recognize related fields, with small tables there is no
> > problem but with millon of records the app wait for a long time to
> > stablish manually the relation.
>
> That is to be expected - millions of records is a lot of work for any driver!
>
> Regards, Dave.
>

> -----Original Message-----
> From: Alejandro D. Burne [mailto:alejandro.dburne@gmail.com]
> Sent: 04 August 2005 15:15
> To: Dave Page
> Cc: pgsql-odbc@postgresql.org
> Subject: Re: Usind database with encoding UNICODE and vovel
>
> Dave, thanks 4 your time;
> * I've installed MDAC 2.8 with same results.
>
> * If I don't set locale on the client it doesn't work, pgadmin either
> (show wrong encoding data)
>
> After connect trough odbc:
> SHOW CLIENT_ENCODING;
> return -> UTF8

So to make it work, you do " SET client_encoding TO 'LATIN10' " ?

> The same command after connect to the LATIN10 db on pgadmin;
> SHOW CLIENT_ENCODING;
> return -> UNICODE

In pgAdmin it will always be unicode, because pgAdmin explicitly sets it
to unicode unless the database is SQL_ASCII. Thats because pgAdmin and
wxWidgets (the framework it's built with) are fully Unicode aware and
can therefore just let the server handle the conversion.

> Try create a LATIN10 db and connect to it to return client encoding.

The default in MSQUERY is UTF8 (which is UNICODE)

> * About MsQuery I was speaking about visual querys design time, not to
> return millons of records.
> It works fine with other db, because it find the fields to make the
> relationship between tables, right now I'm migrating an app from mysql
> to postgres, the same querys, same data on msquery with mysql (myodbc)
> works fine; with psqlodbc hangup trying to relate tables (on visual
> design, without return a single record), If I type the query it works,
> but don't expect final users write a sql sentence =)

Sounds odd - I can design queries here with no difficulties. You haven't
got SQL Tracing or the Mylog switched on have you?

> * What about the 2 chars used to store strange chars?

Thats likely just multibyte characters being shown as single byte.

Regards, Dave

2005/8/4, Dave Page <dpage@vale-housing.co.uk>:

...
> >
> > Dave, thanks 4 your time;
> > * I've installed MDAC 2.8 with same results.
> >
> > * If I don't set locale on the client it doesn't work, pgadmin either
> > (show wrong encoding data)
> >
> > After connect trough odbc:
> > SHOW CLIENT_ENCODING;
> > return -> UTF8
>
> So to make it work, you do " SET client_encoding TO 'LATIN10' " ?

Yes, I choose LATIN10 after odbc connect and things go ok, I'm using
visualfoxpro.

>
> > The same command after connect to the LATIN10 db on pgadmin;
> > SHOW CLIENT_ENCODING;
> > return -> UNICODE
>
> In pgAdmin it will always be unicode, because pgAdmin explicitly sets it
> to unicode unless the database is SQL_ASCII. Thats because pgAdmin and
> wxWidgets (the framework it's built with) are fully Unicode aware and
> can therefore just let the server handle the conversion.
>
> > Try create a LATIN10 db and connect to it to return client encoding.
>
> The default in MSQUERY is UTF8 (which is UNICODE)
>
> > * About MsQuery I was speaking about visual querys design time, not to
> > return millons of records.
> > It works fine with other db, because it find the fields to make the
> > relationship between tables, right now I'm migrating an app from mysql
> > to postgres, the same querys, same data on msquery with mysql (myodbc)
> > works fine; with psqlodbc hangup trying to relate tables (on visual
> > design, without return a single record), If I type the query it works,
> > but don't expect final users write a sql sentence =)
>
> Sounds odd - I can design queries here with no difficulties. You haven't
> got SQL Tracing or the Mylog switched on have you?

No, I've not using any odbc log check. I'm using Excel - Data - Obtain
remote data - New query (translated, I've got it in spanish) which
uses an asistant to obtain data. When I try to include more than one
table appear a message box telling something like "the asistant can't
relate the tables, you must make it manually", after that appear
MsQuery to do it, but make a full join on both tables. If the tables
have 50-100 records there is no problem (100*50=5000 record) but with
large tables just collapse. I fix it making views and selecting the
fields just I need with a specific filter.

>
> > * What about the 2 chars used to store strange chars?
>
> Thats likely just multibyte characters being shown as single byte.
>
> Regards, Dave
>
Gracias, Alejandro.

Hi Dave,

ODBC 08.01.0003 behaves in case of encoding exactly same as ODBC 8.00.01.01 (installed by PostgreSQL 8.0.3 package).

Best Regards,
Josef Springer

 -- the software company --

Orlando-di-Lasso Str. 2
D-85640 Putzbrunn

Tel.	++49(0)89 600 6920
Fax	++49(0)89 600 69220
mailto	Josef.Springer@joops.com
Website	http://www.joops.com

Dave Page wrote:

---

From: Josef Springer [mailto:Josef.Springer@JOOPS.COM]
Sent: 04 August 2005 14:50
To: Dave Page
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

Hi Dave,

i have not tried 08.01.0003 ! From where to get ?

Best Regards,
Josef Springer

Dave Page wrote:

 

---

From: Josef Springer [ mailto:Josef.Springer@JOOPS.COM ]
Sent: 04 August 2005 13:35
To: Dave Page
Cc: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel

He Dave,

as other wrote, there should be an encoding problem with Windows 2000.

My Environment:
Windows 2000 internal version: 5.0.2195; SP 2
Germal locale
PostgreSQL: 8.0.3
ODBC driver: 8.00.01.01 (5. March 2005)   

 

OK, I'll say this again - have you tried 08.01.0003? I have little time to help, and I'm not going to chase bugs that might easily have been fixed already in newer versions of the driver than you are running.

No MDAC
 

The ODBC Driver manager is part of MDAC - you cannot use ODBC on Windows without it.  If you are running the default version that comes with Windows 2000, then you should try upgrading to the latest (which is what works for me on XP)

 

http://msdn.microsoft.com/data/mdac/default.aspx

 

Regards, Dave.

> -----Original Message-----
> From: Alejandro D. Burne [mailto:alejandro.dburne@gmail.com]
> Sent: 04 August 2005 16:11
> To: Dave Page
> Cc: pgsql-odbc@postgresql.org
> Subject: Re: Usind database with encoding UNICODE and vovel
>
> 2005/8/4, Dave Page <dpage@vale-housing.co.uk>:
>
> ...
> > >
> > > Dave, thanks 4 your time;
> > > * I've installed MDAC 2.8 with same results.
> > >
> > > * If I don't set locale on the client it doesn't work,
> pgadmin either
> > > (show wrong encoding data)
> > >
> > > After connect trough odbc:
> > > SHOW CLIENT_ENCODING;
> > > return -> UTF8
> >
> > So to make it work, you do " SET client_encoding TO 'LATIN10' " ?
>
> Yes, I choose LATIN10 after odbc connect and things go ok, I'm using
> visualfoxpro.

OK, great thanks. Seems the sensible thig to try when you think about it
:-)

Regards, Dave.

Dave Page wrote:
>
>
>
>>-----Original Message-----
>>From: Andreas Pflug [mailto:pgadmin@pse-consulting.de]
>>Sent: 03 August 2005 09:28
>>To: Josef Springer
>>Cc: pgsql-odbc@postgresql.org; Dave Page
>>Subject: Re: [ODBC] Usind database with encoding UNICODE and vovel
>>
>>Josef Springer wrote:
>>
>>>Hi anybody,
>>>
>>>Encironment: Win2k, german locale, PostgreSQL database with UNICODE
>>>encoding, using via ODBC
>>>                    Client encoding UNICODE.
>>>
>>>I can read strings containing vovels without problems, But writing
>>>strings containing vovels does not work !
>>>Can anybody assist ?
>>
>>I've been posting a patch earlier this year (January) because
>>I had the
>>same problem on Unicode databases. But this is not the
>>ultimate patch;
>>it will create data errors with non-unicode databases (e.g.
>>Latin-1).
>
>
> OK, well as you know Andreas, my unicode/locale etc. experience is
> pretty much limited to what I've learnt from you and Hiroshi Saito on
> pgAdmin, however, I'm on a roll fixing bugs in psqlODBC at the moment,
> so let's try to sort this :-)

Sadly I don't even have the time to follow this threas thorougly.
IMHO the current pgadmin behaviour should be workable for all situations
(unless data isn't really stored in the encoding as the server encoding
states):

Client encoding is set to UNICODE automatically for all server encodings
excluding SQL_ASCII and MULE_INTERNAL, because for all other encodings
proper conversions exist. For SQL_ASCII and MULE_INTERNAL these are used
as client encoding too, hoping the app knows what to do.

In case the server encoding is wrong, the client will have to have to
play with client encoding itself. In this case, the driver shouldn't try
to do conversions itself, but leave as much as possible to the client.

Regards,
Andreas