Thread: psqlodbclibpq 8_1_03 unicode problem

psqlodbclibpq 8_1_03 unicode problem

From
"Mario A. Soto Cordones"
Date:
Hi

I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
connections, and until the moment works well, but the problem has it with
caracters like:

character Ñ, show like character ?
character ó, show like character ??

that I can make?

Thank you

Mario Soto



Re: psqlodbclibpq 8_1_03 unicode problem

From
Simeó Reig
Date:
> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
> connections, and until the moment works well, but the problem has it with
> caracters like:
>
> character Ñ, show like character ?
> character ó, show like character ??
>
> that I can make?
>
> Thank you

I've the same problem too with spanish and catalan letters, Seems like it
don't support unicode characters.

Thanks a lot



Re: psqlodbclibpq 8_1_03 unicode problem

From
Josef Springer
Date:
I  have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you
>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend




Re: psqlodbclibpq 8_1_03 unicode problem

From
"Joel Fradkin"
Date:
Not sure why your getting that.
Is the database Unicode?
When I was in SQLASCII mine read as ?
The older driver 7.4 reas SQLASCII ok and presented French characters ok,
but newer version did not.
When I converted to a Unicode database I could read the French characters ok
with the new driver.

By the way been up Since Monday no odbc errors yet :)

Joel Fradkin

Wazagua, Inc.
2520 Trailmate Dr
Sarasota, Florida 34243
Tel.  941-753-7111 ext 305

jfradkin@wazagua.com
www.wazagua.com
Powered by Wazagua
Providing you with the latest Web-based technology & advanced tools.
© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
 This email message is for the use of the intended recipient(s) and may
contain confidential and privileged information.  Any unauthorized review,
use, disclosure or distribution is prohibited.  If you are not the intended
recipient, please contact the sender by reply email and delete and destroy
all copies of the original message, including attachments.




-----Original Message-----
From: pgsql-odbc-owner@postgresql.org
[mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
Sent: Wednesday, August 10, 2005 7:58 AM
To: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I  have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you
>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend




---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings


Re: psqlodbclibpq 8_1_03 unicode problem

From
"Dave Page"
Date:
If someone can post a reproducable example, I can look at this. So far though, every example I’ve tried has worked
perfectlyfor me with various clients... 

/D

-----Original Message-----
From: "Josef Springer"<Josef.Springer@JOOPS.COM>
Sent: 10/08/05 13:14:50
To: "pgsql-odbc@postgresql.org"<pgsql-odbc@postgresql.org>
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I  have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you
>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend




---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlodbclibpq 8_1_03 unicode problem

From
"Dave Page"
Date:
-----Original Message-----
From: "Joel Fradkin"<jfradkin@wazagua.com>
Sent: 10/08/05 14:15:33
To: "'Josef Springer'"<Josef.Springer@JOOPS.COM>, "pgsql-odbc@postgresql.org"<pgsql-odbc@postgresql.org>
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

> Is the database Unicode?
> When I was in SQLASCII mine read as ?
> The older driver 7.4 reas SQLASCII ok and
> presented French characters ok,
> but newer version did not.
> When I converted to a Unicode database I
> could read the French characters ok
> with the new driver.

Indeed - SQLASCII often works 'by accident', however there is no guarantee. You must use an encoding that properly
supportsyour data. Unicode should work for most. 

> By the way been up Since Monday no odbc
> errors yet :)

That’s great news :-)

/D

-----Unmodified Original Message-----
Not sure why your getting that.
Is the database Unicode?
When I was in SQLASCII mine read as ?
The older driver 7.4 reas SQLASCII ok and presented French characters ok,
but newer version did not.
When I converted to a Unicode database I could read the French characters ok
with the new driver.

By the way been up Since Monday no odbc errors yet :)

Joel Fradkin

Wazagua, Inc.
2520 Trailmate Dr
Sarasota, Florida 34243
Tel.  941-753-7111 ext 305

jfradkin@wazagua.com
www.wazagua.com
Powered by Wazagua
Providing you with the latest Web-based technology & advanced tools.
© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
 This email message is for the use of the intended recipient(s) and may
contain confidential and privileged information.  Any unauthorized review,
use, disclosure or distribution is prohibited.  If you are not the intended
recipient, please contact the sender by reply email and delete and destroy
all copies of the original message, including attachments.




-----Original Message-----
From: pgsql-odbc-owner@postgresql.org
[mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
Sent: Wednesday, August 10, 2005 7:58 AM
To: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I  have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you
>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend




---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: psqlodbclibpq 8_1_03 unicode problem

From
"Merlin Moncure"
Date:
> Not sure why your getting that.
> Is the database Unicode?
> When I was in SQLASCII mine read as ?
> The older driver 7.4 reas SQLASCII ok and presented French characters
ok,
> but newer version did not.
> When I converted to a Unicode database I could read the French
characters
> ok
> with the new driver.
>
> By the way been up Since Monday no odbc errors yet :)
>
> Joel Fradkin

Great news, Joel!  I think you are over the hump...you just happened to
jump onto the odbc driver at the worst possible moment (during a big
push to update it after a couple of years of relative neglect).

I think it's fair to say you exhibited a certain amount of bravery for
sticking it out over a few difficult weeks.  Now that you have an
application stack you can rely on, you will find that PostgreSQL is an
excellent foundation to build a business around.  The capabilities of
the database far exceed all others in its class.

Merlin

Re: psqlodbclibpq 8_1_03 unicode problem

From
"Joel Fradkin"
Date:
What I had to do was convert the database to Unicode.
The 7.4 driver can read Unicode in SQLASCII, but the 8.0 series does not
read uncode dat from a SQLASCII database.

Specifically I wrote a application in .net that uses the npgsql .net objects
(downloaded from the downloads menu on postgres.org site).
With it you can specify reading from ascii and writing to Unicode.
The conection to the new Unicode database should specify encoding = Unicode
(actual syntax is in their help).
In the net application after reading the string from the aschii database I
move it to a unicode encodeing object (I did not have to actually do the
conversion as the data was already Unicode I believe, it came from MSSQL
server) and write to the connection to the new database which specifies
encoding =Unicode.
I will include the .net source snippet for the conversion as an example:

This is the conversion, note I had to comment most of the example from help
out to work for me.
    Private Function sqltounicode(ByVal asciiString As String) As String
        'Dim unicodeString As String = "This string contains the unicode
character Pi(" & ChrW(&H3A0) & ")"
        ' Create two different encodings.
        Dim ascii As Encoding = Encoding.ASCII
        Dim [unicode] As Encoding = Encoding.Unicode
        'Dim [unicode] As Encoding = Encoding.UTF8
        'Dim myconvertstr
        'myconvertstr = "à"
        ' Convert the string into a byte[].
        Dim unicodeBytes As Byte() = [unicode].GetBytes(asciiString)
        'Dim asciiBytes As Byte() = ascii.GetBytes(asciiString)
        'Dim asciiBytes As Byte() = [unicode].GetBytes(myconvertstr)
        'Dim test1, test2
        'test1 = [unicode].GetString(asciiBytes)
        'test2 = ascii.GetString(asciiBytes)
        ' Perform the conversion from one encoding to the other.
        'Dim unicodeBytes As Byte() = Encoding.Convert(ascii, [unicode],
asciiBytes)
        'test1 = [unicode].GetString(unicodeBytes)
        'test2 = ascii.GetString(unicodeBytes)

        'sqltounicode = [unicode].GetString(unicodeBytes)
        sqltounicode = [unicode].GetString(unicodeBytes)
        ' Convert the new byte[] into a char[] and then into a string.
        ' This is a slightly different approach to converting to illustrate
        ' the use of GetCharCount/GetChars.
        'Dim asciiChars(ascii.GetCharCount(asciiBytes, 0,
asciiBytes.Length)) As Char
        ' Dim unicodeChars([unicode].GetCharCount(unicodeBytes, 0,
unicodeBytes.Length)) As Char
        'ascii.GetChars(asciiBytes, 0, asciiBytes.Length, asciiChars, 0)
        ' [unicode].GetChars(unicodeBytes, 0, unicodeBytes.Length,
unicodeChars, 0)
        ' Dim unicodeString As New String(unicodeChars)
        'sqltounicode = unicodeString
        ' Display the strings created before and after the conversion.
        'Console.WriteLine("Original string: {0}", unicodeString)
        'Console.WriteLine("Ascii converted string: {0}", asciiString)
    End Function


This loops through a set of table names (I used a seq to deal with
dependancies
Private Sub Buttontounicode_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Buttontounicode.Click
        Dim strConnectSource
        Dim insertsqltext As String
        Dim isfirstrow As Boolean
        Dim NeedsQuotes As Boolean
        Dim insertvaluestr As String
        Dim addingfield As Boolean
        Dim SQLconnString2 As String
        'Dim dbconntbl As SqlConnection
        Dim dbcomm2 As NpgsqlCommand 'used to get table names
        Dim dbcomm As New NpgsqlCommand 'used to get data
        'dbcomm = New NpgsqlCommand(strSQL, cnn)
        Dim cmd As New NpgsqlCommand
        Dim tablename As String
        Dim cnn As New NpgsqlConnection("Server=" +
TextBoxpostgresserver.Text + ";Port=5432;User
Id=postgres;Password=;Database=" + TextBoxposgresdbname.Text + ";")
        cnn.Open()
        Dim myreader As NpgsqlDataReader
        Dim cnnunicode As New NpgsqlConnection("Server=" +
TextBoxunicodeserver.Text + ";Port=5432;User
Id=postgres;Password=;Database=" + TextBoxunicodedbname.Text +
";Encoding=UNICODE;")
        cnnunicode.Open()

        Dim i As Integer
        Dim mytype, myfieldname As String
        Dim strSQL, rptsql, datagrid1, datagrid2, datagrid2a, datagrid3,
dataCust, dbread
        Dim teststr
        ' Start a local transaction
        Try
            '    myCommand.CommandText = "SET client_encoding= 'UNICODE';"
            '   myCommand.ExecuteNonQuery()

            '    dbconn.Open()

            strSQL = "Select  name "
            strSQL = strSQL + " from tbl_tablnames  " 'where name ~*
'tblresponse_line'"
            strSQL = strSQL + " order by seq "

            'strSQL = "SELECT tablename FROM pg_tables WHERE tableowner =
current_user and schemaname = 'public'"
            dbcomm2 = New NpgsqlCommand(strSQL, cnn)
            Dim myreader2 As NpgsqlDataReader = dbcomm2.ExecuteReader
            While myreader2.Read()
                tablename = myreader2(0).ToString()
                'tablename = "tblcase"
                strSQL = "SELECT * FROM " + tablename '+ " where clientnum =
'law' and customerid = 41288 "
                'strSQL = "SELECT * FROM tblcase where clientnum = 'HBC' and
casenum ='01613-00082-05'"
                dbcomm.Connection = cnn
                dbcomm.CommandText = strSQL
                'Dim myreader As NpgsqlDataReader = dbcomm.ExecuteReader
                myreader = dbcomm.ExecuteReader
                'Dim myreader As NpgsqlDataReader = dbcomm.ExecuteReader
                While myreader.Read()
                    isfirstrow = True
                    insertsqltext = "insert into " + tablename + "("
                    For i = 0 To myreader.FieldCount() - 1
                        addingfield = True
                        If IsDBNull(myreader(i)) Then
                            addingfield = False
                        End If
                        If addingfield Then
                            teststr = myreader.GetDataTypeName(i)
                            If myreader.GetDataTypeName(i) = "timestamp" Or
myreader.GetDataTypeName(i) = "datetime" Or myreader.GetDataTypeName(i) =
"smalldatetime" Then
                                insertvaluestr = CStr(myreader(i))
                                If Len(insertvaluestr) = 9 Or
Len(insertvaluestr) = 8 Or Len(insertvaluestr) = 11 Or Len(insertvaluestr) =
10 Then
                                    If InStr(insertvaluestr, ":") > 0 Then
                                        addingfield = False
                                    End If
                                End If
                            End If
                        End If
                        If addingfield Then
                            insertvaluestr = myreader.GetName(i).ToString
                            If isfirstrow Then
                                isfirstrow = False
                                insertsqltext = insertsqltext +
insertvaluestr
                            Else
                                insertsqltext = insertsqltext + "," +
insertvaluestr
                            End If
                        End If
                    Next
                    '**********Get values
                    insertsqltext = insertsqltext + ") values("
                    isfirstrow = True
                    For i = 0 To myreader.FieldCount() - 1
                        addingfield = True
                        If IsDBNull(myreader(i)) Then
                            addingfield = False
                        End If
                        If addingfield Then
                            If myreader.GetDataTypeName(i) = "timestamp" Or
myreader.GetDataTypeName(i) = "datetime" Or myreader.GetDataTypeName(i) =
"smalldatetime" Then
                                insertvaluestr = CStr(myreader(i))
                                If Len(insertvaluestr) = 9 Or
Len(insertvaluestr) = 8 Or Len(insertvaluestr) = 11 Or Len(insertvaluestr) =
10 Then
                                    If InStr(insertvaluestr, ":") > 0 Then
                                        addingfield = False
                                    Else
                                        insertvaluestr = insertvaluestr + "
10:00:00"
                                    End If
                                End If
                            End If
                        End If
                        If myreader.GetDataTypeName(i) = "bit" Then
                            NeedsQuotes = False
                        ElseIf myreader.GetDataTypeName(i) = "char" Or
myreader.GetDataTypeName(i) = "ntext" Or myreader.GetDataTypeName(i) =
"text" Or myreader.GetDataTypeName(i) = "varchar" Or
myreader.GetDataTypeName(i) = "nvarchar" Or myreader.GetDataTypeName(i) =
"datetime" Or myreader.GetDataTypeName(i) = "smalldatetime" Then
                            NeedsQuotes = True
                        ElseIf myreader.GetDataTypeName(i) = "decimal" Or
myreader.GetDataTypeName(i) = "float" Or myreader.GetDataTypeName(i) = "int"
Or myreader.GetDataTypeName(i) = "money" Or myreader.GetDataTypeName(i) =
"smallint" Or myreader.GetDataTypeName(i) = "smallmoney" Or
myreader.GetDataTypeName(i) = "tinyint" Then
                            NeedsQuotes = False
                        End If
                        If addingfield Then
                            insertvaluestr =
myreader.GetName(i).ToString.Trim()
                            insertvaluestr = CStr(myreader(i))
                            insertvaluestr = sqltounicode(insertvaluestr)
                            insertvaluestr = insertvaluestr.Replace("'",
"''")
                            insertvaluestr = insertvaluestr.Replace("\",
"\\")
                            insertvaluestr = insertvaluestr.Trim

                            If UCase(myreader.GetName(i).ToString.Trim) =
"ISDELETED" Then
                                If UCase(insertvaluestr) = "1" Then
                                    insertvaluestr = "True"
                                Else
                                    If UCase(insertvaluestr) = "0" Then
                                        insertvaluestr = "False"
                                    End If
                                End If
                            End If

                            If UCase(myreader.GetName(i).ToString.Trim) =
"ACTIVE" Then
                                If UCase(insertvaluestr) = "Y" Then
                                    insertvaluestr = "True"
                                Else
                                    If UCase(insertvaluestr) = "N" Then
                                        insertvaluestr = "False"
                                    End If
                                End If
                            End If
                            'End If

                            If NeedsQuotes Then
                                If isfirstrow Then
                                    isfirstrow = False
                                    insertsqltext = insertsqltext + "'" +
insertvaluestr + "'"
                                Else
                                    insertsqltext = insertsqltext + ",'" +
insertvaluestr + "'"
                                End If
                            Else
                                If isfirstrow Then
                                    isfirstrow = False
                                    insertsqltext = insertsqltext + "" +
insertvaluestr
                                Else
                                    insertsqltext = insertsqltext + "," +
insertvaluestr
                                End If
                            End If
                        End If
                    Next
                    'insert into
tblUser(ClientNum,UserID,LocationID,CompleteName,UserName,PassWord,CreateDat
e,LastModified,IsLocked,Email,OfflineModules,FailedLogins,PermCaseManagement
,PermGeneralInv,PermEmployeInv,PermExceptionReportsTemplates,PermExceptionRe
portsQueue,PermAssociateDatabase,PermEmergencyContact,PermAwardsManagement,P
ermIncidentsManagement,PermSystemAdministration,IsSuperUser,LevelID,PermPara
meterSetting,JobTitleID,CreatedByID,IsDeleted,PermCollectionManagement,Last_
Login,PermSafe,PermRisk,PermClaims,PermAudit,PermPrior,PermORT,PermAuditCrea
tion,PermTraining,PermStats,isCollected,PermShrink,PermExport,DefaultAuditVi
ew,DefaultDisplay,Last_PassWord_Update,UserMovementLogging,PermEthics,Presen
tationID,PermBlackberryAudit,PermEvidenceFolder,PermPresentationLayer,PermTr
ansportSealIssues,PermTimeandBilling,PermFacilitiesManagement,PermSkillInven
tory,PermSubcontractor,PermVendor,PermInternalInventoryTracking,PermParenAud
it) values('LIM',1,1,'Super user','admin','1brain','1/13/2000 3:53:03
PM','4/10/2002 11:21:05
AM',False,'','',0,'1;1;1;1;','1;1;1;1;','1;1;1;1;','0;0;0;0;','0;0;0;0;','1;
1;1;1;','1;1;1;1;','1;1;1;1;1;','1;1;1;1;','1;1;1;1;',False,1,'1;1;1;1;',,,F
alse,'1;1;1;1;','10/30/2004 3:44:15
PM','1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','0;0;0;0;','1;1;1
;1;','0;0;0;0;','1;1;1;1;',True,'1;1;1;1;','1;1;0;0;','S','A
','1/1/2002','N','1;1;1;1;',1,'1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','1
;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;','1;1;1;1;')
                    insertsqltext = insertsqltext + ")"
                    Try
                        'Dim cmd As New NpgsqlCommand(insertsqltext,
cnnunicode)
                        cmd.Connection = cnnunicode
                        'insertsqltext = sqltounicode(insertsqltext)
                        cmd.CommandText = insertsqltext
                        cmd.ExecuteNonQuery()
                    Catch ex As Exception
                        HandleError(ex, insertsqltext)
                        Labelinfo.Text &= "An exception of type " &
ex.GetType().ToString() & "was encountered while inserting the data.:" +
insertsqltext
                    Finally
                    End Try
                End While 'getting data
                'Client = myreader("client")
                myreader.Close()
                'myConnection.Close()
            End While 'getting tables
            myreader = Nothing
            myreader2.Close()
            myreader2 = Nothing
        Catch ex As System.OutOfMemoryException
            HandleError(ex, strSQL)
        Catch ex As System.Data.OleDb.OleDbException
            HandleError(ex, strSQL)
        Catch ex As System.Data.SqlClient.SqlException
            HandleError(ex, strSQL)
        Catch ex As System.Data.Odbc.OdbcException
            HandleError(ex, strSQL)
        Finally
            ' always call Close when done reading.
            'myConnection.Close()
            dbcomm.Dispose()
            dbcomm2.Dispose()
            cmd.Dispose()
            cnnunicode.Close()
            cnnunicode.Dispose()
            cnn.Close()
            cnn.Dispose()
        End Try

    End Sub

Joel Fradkin

Wazagua, Inc.
2520 Trailmate Dr
Sarasota, Florida 34243
Tel.  941-753-7111 ext 305

jfradkin@wazagua.com
www.wazagua.com
Powered by Wazagua
Providing you with the latest Web-based technology & advanced tools.
© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
 This email message is for the use of the intended recipient(s) and may
contain confidential and privileged information.  Any unauthorized review,
use, disclosure or distribution is prohibited.  If you are not the intended
recipient, please contact the sender by reply email and delete and destroy
all copies of the original message, including attachments.




-----Original Message-----
From: Simeó Reig [mailto:simeo@incofisa.com]
Sent: Wednesday, August 10, 2005 10:55 AM
To: Joel Fradkin
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

Hi Joel,

It's seems like this is my problem, I've a SQL_ASCII database ... Is it
possible to change to unicode for read catalan letters from ODBC ? I 've
been reading alter database manual and seems like there is not this
possibility ... How do you made it ?

Thanks

Simeó Reig
Barcelona


----- Original Message -----
From: "Joel Fradkin" <jfradkin@wazagua.com>
To: "'Josef Springer'" <Josef.Springer@JOOPS.COM>;
<pgsql-odbc@postgresql.org>
Sent: Wednesday, August 10, 2005 3:01 PM
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem


Not sure why your getting that.
Is the database Unicode?
When I was in SQLASCII mine read as ?
The older driver 7.4 reas SQLASCII ok and presented French characters ok,
but newer version did not.
When I converted to a Unicode database I could read the French characters ok
with the new driver.

By the way been up Since Monday no odbc errors yet :)

Joel Fradkin

Wazagua, Inc.
2520 Trailmate Dr
Sarasota, Florida 34243
Tel.  941-753-7111 ext 305

jfradkin@wazagua.com
www.wazagua.com
Powered by Wazagua
Providing you with the latest Web-based technology & advanced tools.
© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
 This email message is for the use of the intended recipient(s) and may
contain confidential and privileged information.  Any unauthorized review,
use, disclosure or distribution is prohibited.  If you are not the intended
recipient, please contact the sender by reply email and delete and destroy
all copies of the original message, including attachments.




-----Original Message-----
From: pgsql-odbc-owner@postgresql.org
[mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
Sent: Wednesday, August 10, 2005 7:58 AM
To: pgsql-odbc@postgresql.org
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I  have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

>> I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>> connections, and until the moment works well, but the problem has it
>> with
>> caracters like:
>>
>> character Ñ, show like character ?
>> character ó, show like character ??
>>
>> that I can make?
>>
>> Thank you
>
>
> I've the same problem too with spanish and catalan letters, Seems like
> it don't support unicode characters.
>
> Thanks a lot
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend




---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend



Re: psqlodbclibpq 8_1_03 unicode problem

From
"Joel Fradkin"
Date:
Well my name is mud at the moment (had a huge problem during my conversion
related to a bad backup that I did not catch until too late).

Hopefully the upcoming year on what is a big step in the right direction
will wash some of the mud away and my client base will be happy in the end.

I do not know if it is my imagination, but the servers seem to be running
with better response times then before, but from what I read on the test ran
by folks who were benchmarking I thought the new driver was a tad slower
then the old one.

Could be the problems I was having with the odbc causing the crashes etc
were in some way impacting the performance and even though the new drivers
are supposed to be slower in my case there was an increase in performance.

In any case as long as I do not see a catastrophic error or an odbc related
issue I will be dancing for joy.

I can not thank everyone who has made contributions on this enough. I know
it was a great many folks, so to each and every one who helped thanks a
bunch!

Joel Fradkin


-----Original Message-----
From: Merlin Moncure [mailto:merlin.moncure@rcsonline.com]
Sent: Wednesday, August 10, 2005 12:07 PM
To: Joel Fradkin
Cc: pgsql-odbc@postgresql.org
Subject: RE: [ODBC] psqlodbclibpq 8_1_03 unicode problem

> Not sure why your getting that.
> Is the database Unicode?
> When I was in SQLASCII mine read as ?
> The older driver 7.4 reas SQLASCII ok and presented French characters
ok,
> but newer version did not.
> When I converted to a Unicode database I could read the French
characters
> ok
> with the new driver.
>
> By the way been up Since Monday no odbc errors yet :)
>
> Joel Fradkin

Great news, Joel!  I think you are over the hump...you just happened to
jump onto the odbc driver at the worst possible moment (during a big
push to update it after a couple of years of relative neglect).

I think it's fair to say you exhibited a certain amount of bravery for
sticking it out over a few difficult weeks.  Now that you have an
application stack you can rely on, you will find that PostgreSQL is an
excellent foundation to build a business around.  The capabilities of
the database far exceed all others in its class.

Merlin


Re: psqlodbclibpq 8_1_03 unicode problem

From
Josef Springer
Date:
Hi Dave,

sorry i can't send you an example, because we use PostgreSQL with a Smalltalk environment. For an example you need not only the example itself but the most development environment. I have reported allread a bug for this.

Josef Spinger

Dave Page wrote:
If someone can post a reproducable example, I can look at this. So far though, every example I’ve tried has worked perfectly for me with various clients...

/D

-----Original Message-----
From: "Josef Springer"<Josef.Springer@JOOPS.COM>
Sent: 10/08/05 13:14:50
To: "pgsql-odbc@postgresql.org"<pgsql-odbc@postgresql.org>
Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem

I have the same problem and reported a bug.
It seems, that Unicode is readed unencoded (one byte per character)

Josef Springer

Simeó Reig wrote:

I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
connections, and until the moment works well, but the problem has it
with
caracters like:

character Ñ, show like character ?
character ó, show like character ??

that I can make?

Thank you

I've the same problem too with spanish and catalan letters, Seems like
it don't support unicode characters.

Thanks a lot


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend




---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlodbclibpq 8_1_03 unicode problem

From
Marko Ristola
Date:
Characters will go into question marks, if the character conversion is
not defined
properly, or the input for the conversion functions is incorrect.


Here is something about ANSI C standard about the character conversions:


setlocale(LC_ALL,"C");
After calling this function in main() C program, the program
works as follows:

LATIN1 works with itself, because it does not need any conversions and
it is single byte
encoding.
UTF-8 works with itself, because it does not need any locale specific
information to work.

Conversion from UTF-8 into LATIN 1does not work. Non-ascii characters
will be converted
into question marks.
Conversion from LATIN1 into UTF-8 does not work. Non-ascii characters
will be converted
into question marks.

setlocale(LC_ALL,""); or setlocale(LC_CTYPE,"");

ANSI C definition for this is, that locale environment settings (Windows
settings, or Linux environment settings), for example
LC_CTYPE=fi_FI.iso8859-1 is used. This way conversions between LATIN1
and UTF-8 will work, if the environment is ANSI C compilant (it should).

So the LC_CTYPE must match with the multibyte data. Windows settings
will be activated
with the "" setting: an empty string means the environment default.

There is a good book about ANSI C 99: "The C standard"


So, how this relates to psqlodbc?

Case 1:
Typically user needs to do the conversions outside US.
Inside psqlodbc, do setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
and do setlocale(LC_CTYPE,"C");

Case 2:
Sometimes user does not want to do any charset conversions
because of performance reasons.
Don't support this one.

Case 3:
User has already done setlocale(LC_CTYPE,"");
To make Case 1 and Case 3 work, we do inside ODBC the following:
strncpy(prev_locale,setlocale(LC_CTYPE,NULL),64);
setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
and do setlocale(LC_CTYPE,prev_locale);

Case 4:
A threaded application needs to work also.
Under Windows, setlocale() can be altered to behave as thread local, so
that every
thread has it's own locale setting.
Under Linux, at least iconv() conversion facility works, even though it
is not ANSI C.


Regards, Marko Ristola



Joel Fradkin wrote:

>Not sure why your getting that.
>Is the database Unicode?
>When I was in SQLASCII mine read as ?
>The older driver 7.4 reas SQLASCII ok and presented French characters ok,
>but newer version did not.
>When I converted to a Unicode database I could read the French characters ok
>with the new driver.
>
>By the way been up Since Monday no odbc errors yet :)
>
>Joel Fradkin
>
>Wazagua, Inc.
>2520 Trailmate Dr
>Sarasota, Florida 34243
>Tel.  941-753-7111 ext 305
>
>jfradkin@wazagua.com
>www.wazagua.com
>Powered by Wazagua
>Providing you with the latest Web-based technology & advanced tools.
>© 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
> This email message is for the use of the intended recipient(s) and may
>contain confidential and privileged information.  Any unauthorized review,
>use, disclosure or distribution is prohibited.  If you are not the intended
>recipient, please contact the sender by reply email and delete and destroy
>all copies of the original message, including attachments.
>
>
>
>
>-----Original Message-----
>From: pgsql-odbc-owner@postgresql.org
>[mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
>Sent: Wednesday, August 10, 2005 7:58 AM
>To: pgsql-odbc@postgresql.org
>Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem
>
>I  have the same problem and reported a bug.
>It seems, that Unicode is readed unencoded (one byte per character)
>
>Josef Springer
>
>Simeó Reig wrote:
>
>
>
>>>I am using the versionde psqlodbclibpq 8_01_003, to be able to use ssl
>>>connections, and until the moment works well, but the problem has it
>>>with
>>>caracters like:
>>>
>>>character Ñ, show like character ?
>>>character ó, show like character ??
>>>
>>>that I can make?
>>>
>>>Thank you
>>>
>>>
>>I've the same problem too with spanish and catalan letters, Seems like
>>it don't support unicode characters.
>>
>>Thanks a lot
>>
>>
>>---------------------------(end of broadcast)---------------------------
>>TIP 6: explain analyze is your friend
>>
>>
>
>
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: don't forget to increase your free space map settings
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: explain analyze is your friend
>
>


Re: psqlodbclibpq 8_1_03 unicode problem

From
"Mario A. Soto Cordones"
Date:
Hi

I find this mail very well, but I don't see I eat up he can help with the
psqlodbc problem.

thank you



>
> Characters will go into question marks, if the character conversion is
> not defined
> properly, or the input for the conversion functions is incorrect.
>
>
> Here is something about ANSI C standard about the character conversions:
>
>
> setlocale(LC_ALL,"C");
> After calling this function in main() C program, the program
> works as follows:
>
> LATIN1 works with itself, because it does not need any conversions and
> it is single byte
> encoding.
> UTF-8 works with itself, because it does not need any locale specific
> information to work.
>
> Conversion from UTF-8 into LATIN 1does not work. Non-ascii characters
> will be converted
> into question marks.
> Conversion from LATIN1 into UTF-8 does not work. Non-ascii characters
> will be converted
> into question marks.
>
> setlocale(LC_ALL,""); or setlocale(LC_CTYPE,"");
>
> ANSI C definition for this is, that locale environment settings (Windows
> settings, or Linux environment settings), for example
> LC_CTYPE=fi_FI.iso8859-1 is used. This way conversions between LATIN1
> and UTF-8 will work, if the environment is ANSI C compilant (it should).
>
> So the LC_CTYPE must match with the multibyte data. Windows settings
> will be activated
> with the "" setting: an empty string means the environment default.
>
> There is a good book about ANSI C 99: "The C standard"
>
>
> So, how this relates to psqlodbc?
>
> Case 1:
> Typically user needs to do the conversions outside US.
> Inside psqlodbc, do setlocale(LC_CTYPE,""), do the LATIN1/UTF-8
> -conversion and do setlocale(LC_CTYPE,"C");
>
> Case 2:
> Sometimes user does not want to do any charset conversions
> because of performance reasons.
> Don't support this one.
>
> Case 3:
> User has already done setlocale(LC_CTYPE,"");
> To make Case 1 and Case 3 work, we do inside ODBC the following:
> strncpy(prev_locale,setlocale(LC_CTYPE,NULL),64);
> setlocale(LC_CTYPE,""), do the LATIN1/UTF-8 -conversion
> and do setlocale(LC_CTYPE,prev_locale);
>
> Case 4:
> A threaded application needs to work also.
> Under Windows, setlocale() can be altered to behave as thread local, so
> that every
> thread has it's own locale setting.
> Under Linux, at least iconv() conversion facility works, even though it
> is not ANSI C.
>
>
> Regards, Marko Ristola
>
>
>
> Joel Fradkin wrote:
>
>>Not sure why your getting that.
>>Is the database Unicode?
>>When I was in SQLASCII mine read as ?
>>The older driver 7.4 reas SQLASCII ok and presented French characters
>> ok, but newer version did not.
>>When I converted to a Unicode database I could read the French
>> characters ok with the new driver.
>>
>>By the way been up Since Monday no odbc errors yet :)
>>
>>Joel Fradkin
>>
>>Wazagua, Inc.
>>2520 Trailmate Dr
>>Sarasota, Florida 34243
>>Tel.  941-753-7111 ext 305
>>
>>jfradkin@wazagua.com
>>www.wazagua.com
>>Powered by Wazagua
>>Providing you with the latest Web-based technology & advanced tools. ©
>> 2004. WAZAGUA, Inc. All rights reserved. WAZAGUA, Inc
>> This email message is for the use of the intended recipient(s) and may
>>contain confidential and privileged information.  Any unauthorized
>> review, use, disclosure or distribution is prohibited.  If you are not
>> the intended recipient, please contact the sender by reply email and
>> delete and destroy all copies of the original message, including
>> attachments.
>>
>>
>>
>>
>>-----Original Message-----
>>From: pgsql-odbc-owner@postgresql.org
>>[mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Josef Springer
>> Sent: Wednesday, August 10, 2005 7:58 AM
>>To: pgsql-odbc@postgresql.org
>>Subject: Re: [ODBC] psqlodbclibpq 8_1_03 unicode problem
>>
>>I  have the same problem and reported a bug.
>>It seems, that Unicode is readed unencoded (one byte per character)
>>
>>Josef Springer
>>
>>Simeó Reig wrote:
>>
>>
>>
>>>>I am using the versionde psqlodbclibpq 8_01_003, to be able to use
>>>> ssl connections, and until the moment works well, but the problem has
>>>> it  with
>>>>caracters like:
>>>>
>>>>character Ñ, show like character ?
>>>>character ó, show like character ??
>>>>
>>>>that I can make?
>>>>
>>>>Thank you
>>>>
>>>>
>>>I've the same problem too with spanish and catalan letters, Seems like
>>>  it don't support unicode characters.
>>>
>>>Thanks a lot
>>>
>>>
>>>---------------------------(end of
>>> broadcast)--------------------------- TIP 6: explain analyze is your
>>> friend
>>>
>>>
>>
>>
>>
>>
>>---------------------------(end of
>> broadcast)--------------------------- TIP 5: don't forget to increase
>> your free space map settings
>>
>>
>>---------------------------(end of
>> broadcast)--------------------------- TIP 6: explain analyze is your
>> friend
>>
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
> match