Thread: [ODBC] ODBC application works while libpq fails

[ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

31 July 2017, 19:05:37

 Hi,
I have a very weird situation.

I am trying to develop an application which will connect to the PostgreSQL
server thru 2 different methods: thru the ODBC and using libpq.

Environment:
MacOSX 10.8.
Xcode 4 (latest available for 10.8)
PostgreSQL 9.1 - original from OSX.
Application is compiled with the UNICODE
iODBC DM for OSX 10.8.
Both application and DB server running on the same machine.
The database I'm using was created with the default parameters (just
CREATE DATABASE draft).

When I connect with the ODBC driver, the application works as expected
- no errors or problems.

However when I tried to run the application with libpq I received an
error. The error is:

[quote]
Invalid byte sequence for encoding UTF-8 0xc3\n
[/quote]

The application is written in C++ (not ObjC++).
The table I am trying to query contains the field with the word
"abc+<ALT225>", which inside the database is translated as
"abc\xc3\x9f".

What could be the reason for such behavior?

Is ODBC driver sets the client encoding to some uniform value?
Or there something else?

Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

02 August 2017, 09:40:46

Hi,

On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>  Hi,
> I have a very weird situation.
>
> I am trying to develop an application which will connect to the PostgreSQL
> server thru 2 different methods: thru the ODBC and using libpq.
>
> Environment:
> MacOSX 10.8.
> Xcode 4 (latest available for 10.8)
> PostgreSQL 9.1 - original from OSX.
> Application is compiled with the UNICODE
> iODBC DM for OSX 10.8.
> Both application and DB server running on the same machine.
> The database I'm using was created with the default parameters (just
> CREATE DATABASE draft).
>
> When I connect with the ODBC driver, the application works as expected
> - no errors or problems.
>
> However when I tried to run the application with libpq I received an
> error. The error is:
>
> [quote]
> Invalid byte sequence for encoding UTF-8 0xc3\n
> [/quote]
>
> The application is written in C++ (not ObjC++).
> The table I am trying to query contains the field with the word
> "abc+<ALT225>", which inside the database is translated as
> "abc\xc3\x9f".
>
> What could be the reason for such behavior?
>
> Is ODBC driver sets the client encoding to some uniform value?
> Or there something else?

Here is the content of my .odbc.ini for PostgreSQL on Mac:

[code]
[postgre]
Driver               = PostgreSQL Unicode Driver
host                 = localhost
ServerName           = localhost
SERVER               = localhost
PORT                 = 5432
UserName             = postgres
Password             = <my_passwd>
Database             = draft
UseServerSidePrepare = 1
[/code]

As you can see there is no special encoding settings. So it uses one
that is set by default.
But using ODBC API works and trying libpq - doesn't.

Could someone please help?

Thank you.


>
> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

"Inoue, Hiroshi"

Date:

02 August 2017, 10:32:31

Hi Igor,

On 2017/08/02 12:40, Igor Korot wrote:
> Hi,
>
> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>   Hi,
>> I have a very weird situation.
>>
>> I am trying to develop an application which will connect to the PostgreSQL
>> server thru 2 different methods: thru the ODBC and using libpq.
>>
>> Environment:
>> MacOSX 10.8.
>> Xcode 4 (latest available for 10.8)
>> PostgreSQL 9.1 - original from OSX.
>> Application is compiled with the UNICODE
>> iODBC DM for OSX 10.8.
>> Both application and DB server running on the same machine.
>> The database I'm using was created with the default parameters (just
>> CREATE DATABASE draft).
>>
>> When I connect with the ODBC driver, the application works as expected
>> - no errors or problems.
>>
>> However when I tried to run the application with libpq I received an
>> error. The error is:
>>
>> [quote]
>> Invalid byte sequence for encoding UTF-8 0xc3\n
>> [/quote]
>>
>> The application is written in C++ (not ObjC++).
>> The table I am trying to query contains the field with the word
>> "abc+<ALT225>", which inside the database is translated as
>> "abc\xc3\x9f".
>>
>> What could be the reason for such behavior?
>>
>> Is ODBC driver sets the client encoding to some uniform value?
>> Or there something else?
> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>
> [code]
> [postgre]
> Driver               = PostgreSQL Unicode Driver
> host                 = localhost
> ServerName           = localhost
> SERVER               = localhost
> PORT                 = 5432
> UserName             = postgres
> Password             = <my_passwd>
> Database             = draft
> UseServerSidePrepare = 1
> [/code]
>
> As you can see there is no special encoding settings. So it uses one
> that is set by default.
> But using ODBC API works and trying libpq - doesn't.
>
> Could someone please help?

What is a query which causes a failure?
What is a result of locale command?

regards,
Hiroshi Inoue

>
> Thank you.
>
>
>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

03 August 2017, 10:06:11

Hi, Hiroshi,

On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
<h-inoue@dream.email.ne.jp> wrote:
> Hi Igor,
>
>
> On 2017/08/02 12:40, Igor Korot wrote:
>>
>> Hi,
>>
>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>
>>>   Hi,
>>> I have a very weird situation.
>>>
>>> I am trying to develop an application which will connect to the
>>> PostgreSQL
>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>
>>> Environment:
>>> MacOSX 10.8.
>>> Xcode 4 (latest available for 10.8)
>>> PostgreSQL 9.1 - original from OSX.
>>> Application is compiled with the UNICODE
>>> iODBC DM for OSX 10.8.
>>> Both application and DB server running on the same machine.
>>> The database I'm using was created with the default parameters (just
>>> CREATE DATABASE draft).
>>>
>>> When I connect with the ODBC driver, the application works as expected
>>> - no errors or problems.
>>>
>>> However when I tried to run the application with libpq I received an
>>> error. The error is:
>>>
>>> [quote]
>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>> [/quote]
>>>
>>> The application is written in C++ (not ObjC++).
>>> The table I am trying to query contains the field with the word
>>> "abc+<ALT225>", which inside the database is translated as
>>> "abc\xc3\x9f".
>>>
>>> What could be the reason for such behavior?
>>>
>>> Is ODBC driver sets the client encoding to some uniform value?
>>> Or there something else?
>>
>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>
>> [code]
>> [postgre]
>> Driver               = PostgreSQL Unicode Driver
>> host                 = localhost
>> ServerName           = localhost
>> SERVER               = localhost
>> PORT                 = 5432
>> UserName             = postgres
>> Password             = <my_passwd>
>> Database             = draft
>> UseServerSidePrepare = 1
>> [/code]
>>
>> As you can see there is no special encoding settings. So it uses one
>> that is set by default.
>> But using ODBC API works and trying libpq - doesn't.
>>
>> Could someone please help?
>
>
> What is a query which causes a failure?
> What is a result of locale command?

Sorry for the noise.
I think I found the culprit and in the process of fixing it.

I will let you know how it goes.

Thank you.

>
> regards,
> Hiroshi Inoue
>
>>
>> Thank you.
>>
>>
>>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

04 August 2017, 17:59:45

Hi, Hiroshi,

On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
> Hi, Hiroshi,
>
> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
> <h-inoue@dream.email.ne.jp> wrote:
>> Hi Igor,
>>
>>
>> On 2017/08/02 12:40, Igor Korot wrote:
>>>
>>> Hi,
>>>
>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>
>>>>   Hi,
>>>> I have a very weird situation.
>>>>
>>>> I am trying to develop an application which will connect to the
>>>> PostgreSQL
>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>
>>>> Environment:
>>>> MacOSX 10.8.
>>>> Xcode 4 (latest available for 10.8)
>>>> PostgreSQL 9.1 - original from OSX.
>>>> Application is compiled with the UNICODE
>>>> iODBC DM for OSX 10.8.
>>>> Both application and DB server running on the same machine.
>>>> The database I'm using was created with the default parameters (just
>>>> CREATE DATABASE draft).
>>>>
>>>> When I connect with the ODBC driver, the application works as expected
>>>> - no errors or problems.
>>>>
>>>> However when I tried to run the application with libpq I received an
>>>> error. The error is:
>>>>
>>>> [quote]
>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>> [/quote]
>>>>
>>>> The application is written in C++ (not ObjC++).
>>>> The table I am trying to query contains the field with the word
>>>> "abc+<ALT225>", which inside the database is translated as
>>>> "abc\xc3\x9f".
>>>>
>>>> What could be the reason for such behavior?
>>>>
>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>> Or there something else?
>>>
>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>
>>> [code]
>>> [postgre]
>>> Driver               = PostgreSQL Unicode Driver
>>> host                 = localhost
>>> ServerName           = localhost
>>> SERVER               = localhost
>>> PORT                 = 5432
>>> UserName             = postgres
>>> Password             = <my_passwd>
>>> Database             = draft
>>> UseServerSidePrepare = 1
>>> [/code]
>>>
>>> As you can see there is no special encoding settings. So it uses one
>>> that is set by default.
>>> But using ODBC API works and trying libpq - doesn't.
>>>
>>> Could someone please help?
>>
>>
>> What is a query which causes a failure?
>> What is a result of locale command?
>
> Sorry for the noise.
> I think I found the culprit and in the process of fixing it.
>
> I will let you know how it goes.

Apparently I didn't fix it yet.

So here is information requested:

MyMac:dbhandler igorkorot$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

And the failing query is:

SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
$2 AND \"abc_cnam\" = $3;

First parameter is "public.abc\xc3\x9f", second - "public" and third - "id".

The error is "Invalid key sequence for UTF-8".

Thank you.

>
> Thank you.
>
>>
>> regards,
>> Hiroshi Inoue
>>
>>>
>>> Thank you.
>>>
>>>
>>>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

"Inoue, Hiroshi"

Date:

04 August 2017, 19:17:09

Hi Igor,

On 2017/08/04 20:59, Igor Korot wrote:
> Hi, Hiroshi,
>
> On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
>> Hi, Hiroshi,
>>
>> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
>> <h-inoue@dream.email.ne.jp> wrote:
>>> Hi Igor,
>>>
>>>
>>> On 2017/08/02 12:40, Igor Korot wrote:
>>>> Hi,
>>>>
>>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>>    Hi,
>>>>> I have a very weird situation.
>>>>>
>>>>> I am trying to develop an application which will connect to the
>>>>> PostgreSQL
>>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>>
>>>>> Environment:
>>>>> MacOSX 10.8.
>>>>> Xcode 4 (latest available for 10.8)
>>>>> PostgreSQL 9.1 - original from OSX.
>>>>> Application is compiled with the UNICODE
>>>>> iODBC DM for OSX 10.8.
>>>>> Both application and DB server running on the same machine.
>>>>> The database I'm using was created with the default parameters (just
>>>>> CREATE DATABASE draft).
>>>>>
>>>>> When I connect with the ODBC driver, the application works as expected
>>>>> - no errors or problems.
>>>>>
>>>>> However when I tried to run the application with libpq I received an
>>>>> error. The error is:
>>>>>
>>>>> [quote]
>>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>>> [/quote]
>>>>>
>>>>> The application is written in C++ (not ObjC++).
>>>>> The table I am trying to query contains the field with the word
>>>>> "abc+<ALT225>", which inside the database is translated as
>>>>> "abc\xc3\x9f".
>>>>>
>>>>> What could be the reason for such behavior?
>>>>>
>>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>>> Or there something else?
>>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>>
>>>> [code]
>>>> [postgre]
>>>> Driver               = PostgreSQL Unicode Driver
>>>> host                 = localhost
>>>> ServerName           = localhost
>>>> SERVER               = localhost
>>>> PORT                 = 5432
>>>> UserName             = postgres
>>>> Password             = <my_passwd>
>>>> Database             = draft
>>>> UseServerSidePrepare = 1
>>>> [/code]
>>>>
>>>> As you can see there is no special encoding settings. So it uses one
>>>> that is set by default.
>>>> But using ODBC API works and trying libpq - doesn't.
>>>>
>>>> Could someone please help?
>>>
>>> What is a query which causes a failure?
>>> What is a result of locale command?
>> Sorry for the noise.
>> I think I found the culprit and in the process of fixing it.
>>
>> I will let you know how it goes.
> Apparently I didn't fix it yet.
>
> So here is information requested:
>
> MyMac:dbhandler igorkorot$ locale
> LANG="en_US.UTF-8"
> LC_COLLATE="en_US.UTF-8"
> LC_CTYPE="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME="en_US.UTF-8"
> LC_ALL=
>
> And the failing query is:
>
> SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
> $2 AND \"abc_cnam\" = $3;
>
> First parameter is "public.abc\xc3\x9f", second - "public" and third - "id".
>
> The error is "Invalid key sequence for UTF-8".

Do you call setlocale(LC_ALL, "") first?

regards,
Hiroshi Inoue
>
> Thank you.
>
>> Thank you.
>>
>>> regards,
>>> Hiroshi Inoue
>>>
>>>> Thank you.
>>>>
>>>>
>>>>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

04 August 2017, 19:22:50

HI, Hiroshi,

On Fri, Aug 4, 2017 at 9:17 AM, Inoue, Hiroshi
<h-inoue@dream.email.ne.jp> wrote:
> Hi Igor,
>
>
> On 2017/08/04 20:59, Igor Korot wrote:
>>
>> Hi, Hiroshi,
>>
>> On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>
>>> Hi, Hiroshi,
>>>
>>> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
>>> <h-inoue@dream.email.ne.jp> wrote:
>>>>
>>>> Hi Igor,
>>>>
>>>>
>>>> On 2017/08/02 12:40, Igor Korot wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>>>
>>>>>>    Hi,
>>>>>> I have a very weird situation.
>>>>>>
>>>>>> I am trying to develop an application which will connect to the
>>>>>> PostgreSQL
>>>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>>>
>>>>>> Environment:
>>>>>> MacOSX 10.8.
>>>>>> Xcode 4 (latest available for 10.8)
>>>>>> PostgreSQL 9.1 - original from OSX.
>>>>>> Application is compiled with the UNICODE
>>>>>> iODBC DM for OSX 10.8.
>>>>>> Both application and DB server running on the same machine.
>>>>>> The database I'm using was created with the default parameters (just
>>>>>> CREATE DATABASE draft).
>>>>>>
>>>>>> When I connect with the ODBC driver, the application works as expected
>>>>>> - no errors or problems.
>>>>>>
>>>>>> However when I tried to run the application with libpq I received an
>>>>>> error. The error is:
>>>>>>
>>>>>> [quote]
>>>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>>>> [/quote]
>>>>>>
>>>>>> The application is written in C++ (not ObjC++).
>>>>>> The table I am trying to query contains the field with the word
>>>>>> "abc+<ALT225>", which inside the database is translated as
>>>>>> "abc\xc3\x9f".
>>>>>>
>>>>>> What could be the reason for such behavior?
>>>>>>
>>>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>>>> Or there something else?
>>>>>
>>>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>>>
>>>>> [code]
>>>>> [postgre]
>>>>> Driver               = PostgreSQL Unicode Driver
>>>>> host                 = localhost
>>>>> ServerName           = localhost
>>>>> SERVER               = localhost
>>>>> PORT                 = 5432
>>>>> UserName             = postgres
>>>>> Password             = <my_passwd>
>>>>> Database             = draft
>>>>> UseServerSidePrepare = 1
>>>>> [/code]
>>>>>
>>>>> As you can see there is no special encoding settings. So it uses one
>>>>> that is set by default.
>>>>> But using ODBC API works and trying libpq - doesn't.
>>>>>
>>>>> Could someone please help?
>>>>
>>>>
>>>> What is a query which causes a failure?
>>>> What is a result of locale command?
>>>
>>> Sorry for the noise.
>>> I think I found the culprit and in the process of fixing it.
>>>
>>> I will let you know how it goes.
>>
>> Apparently I didn't fix it yet.
>>
>> So here is information requested:
>>
>> MyMac:dbhandler igorkorot$ locale
>> LANG="en_US.UTF-8"
>> LC_COLLATE="en_US.UTF-8"
>> LC_CTYPE="en_US.UTF-8"
>> LC_MESSAGES="en_US.UTF-8"
>> LC_MONETARY="en_US.UTF-8"
>> LC_NUMERIC="en_US.UTF-8"
>> LC_TIME="en_US.UTF-8"
>> LC_ALL=
>>
>> And the failing query is:
>>
>> SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
>> $2 AND \"abc_cnam\" = $3;
>>
>> First parameter is "public.abc\xc3\x9f", second - "public" and third -
>> "id".
>>
>> The error is "Invalid key sequence for UTF-8".
>
>
> Do you call setlocale(LC_ALL, "") first?

No I do not.
Is it what ODBC driver does?

Thank you.

BTW, with ODBC the table name is successfully converted to be "abc<ALT+225>"
(it's a greek letter "beta" or a symbol for a German "ss" sound.- ß).


>
>
> regards,
> Hiroshi Inoue
>>
>>
>> Thank you.
>>
>>> Thank you.
>>>
>>>> regards,
>>>> Hiroshi Inoue
>>>>
>>>>> Thank you.
>>>>>
>>>>>
>>>>>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

"Inoue, Hiroshi"

Date:

05 August 2017, 05:22:28

Hi Igor,

On 2017/08/04 22:22, Igor Korot wrote:
> HI, Hiroshi,
>
> On Fri, Aug 4, 2017 at 9:17 AM, Inoue, Hiroshi
> <h-inoue@dream.email.ne.jp> wrote:
>> Hi Igor,
>>
>>
>> On 2017/08/04 20:59, Igor Korot wrote:
>>> Hi, Hiroshi,
>>>
>>> On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>> Hi, Hiroshi,
>>>>
>>>> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
>>>> <h-inoue@dream.email.ne.jp> wrote:
>>>>> Hi Igor,
>>>>>
>>>>>
>>>>> On 2017/08/02 12:40, Igor Korot wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>>>>     Hi,
>>>>>>> I have a very weird situation.
>>>>>>>
>>>>>>> I am trying to develop an application which will connect to the
>>>>>>> PostgreSQL
>>>>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>>>>
>>>>>>> Environment:
>>>>>>> MacOSX 10.8.
>>>>>>> Xcode 4 (latest available for 10.8)
>>>>>>> PostgreSQL 9.1 - original from OSX.
>>>>>>> Application is compiled with the UNICODE
>>>>>>> iODBC DM for OSX 10.8.
>>>>>>> Both application and DB server running on the same machine.
>>>>>>> The database I'm using was created with the default parameters (just
>>>>>>> CREATE DATABASE draft).
>>>>>>>
>>>>>>> When I connect with the ODBC driver, the application works as expected
>>>>>>> - no errors or problems.
>>>>>>>
>>>>>>> However when I tried to run the application with libpq I received an
>>>>>>> error. The error is:
>>>>>>>
>>>>>>> [quote]
>>>>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>>>>> [/quote]
>>>>>>>
>>>>>>> The application is written in C++ (not ObjC++).
>>>>>>> The table I am trying to query contains the field with the word
>>>>>>> "abc+<ALT225>", which inside the database is translated as
>>>>>>> "abc\xc3\x9f".
>>>>>>>
>>>>>>> What could be the reason for such behavior?
>>>>>>>
>>>>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>>>>> Or there something else?
>>>>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>>>>
>>>>>> [code]
>>>>>> [postgre]
>>>>>> Driver               = PostgreSQL Unicode Driver
>>>>>> host                 = localhost
>>>>>> ServerName           = localhost
>>>>>> SERVER               = localhost
>>>>>> PORT                 = 5432
>>>>>> UserName             = postgres
>>>>>> Password             = <my_passwd>
>>>>>> Database             = draft
>>>>>> UseServerSidePrepare = 1
>>>>>> [/code]
>>>>>>
>>>>>> As you can see there is no special encoding settings. So it uses one
>>>>>> that is set by default.
>>>>>> But using ODBC API works and trying libpq - doesn't.
>>>>>>
>>>>>> Could someone please help?
>>>>>
>>>>> What is a query which causes a failure?
>>>>> What is a result of locale command?
>>>> Sorry for the noise.
>>>> I think I found the culprit and in the process of fixing it.
>>>>
>>>> I will let you know how it goes.
>>> Apparently I didn't fix it yet.
>>>
>>> So here is information requested:
>>>
>>> MyMac:dbhandler igorkorot$ locale
>>> LANG="en_US.UTF-8"
>>> LC_COLLATE="en_US.UTF-8"
>>> LC_CTYPE="en_US.UTF-8"
>>> LC_MESSAGES="en_US.UTF-8"
>>> LC_MONETARY="en_US.UTF-8"
>>> LC_NUMERIC="en_US.UTF-8"
>>> LC_TIME="en_US.UTF-8"
>>> LC_ALL=
>>>
>>> And the failing query is:
>>>
>>> SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
>>> $2 AND \"abc_cnam\" = $3;
>>>
>>> First parameter is "public.abc\xc3\x9f", second - "public" and third -
>>> "id".
>>>
>>> The error is "Invalid key sequence for UTF-8".
>>
>> Do you call setlocale(LC_ALL, "") first?
> No I do not.
> Is it what ODBC driver does?

Yes.
Unless setlocale() is called, "C" locale is selected.

regards,
Hiroshi Inoue

> Thank you.
>
> BTW, with ODBC the table name is successfully converted to be "abc<ALT+225>"
> (it's a greek letter "beta" or a symbol for a German "ss" sound.- ß).
>
>
>> regards,
>> Hiroshi Inoue
>>>
>>> Thank you.
>>>
>>>> Thank you.
>>>>
>>>>> regards,
>>>>> Hiroshi Inoue
>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>>
>>>>>>> Thank you.

Re: [ODBC] ODBC application works while libpq fails

From

Igor Korot

Date:

05 August 2017, 10:18:42

Hiroshi,

On Fri, Aug 4, 2017 at 7:22 PM, Inoue, Hiroshi
<h-inoue@dream.email.ne.jp> wrote:
> Hi Igor,
>
>
> On 2017/08/04 22:22, Igor Korot wrote:
>>
>> HI, Hiroshi,
>>
>> On Fri, Aug 4, 2017 at 9:17 AM, Inoue, Hiroshi
>> <h-inoue@dream.email.ne.jp> wrote:
>>>
>>> Hi Igor,
>>>
>>>
>>> On 2017/08/04 20:59, Igor Korot wrote:
>>>>
>>>> Hi, Hiroshi,
>>>>
>>>> On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>>
>>>>> Hi, Hiroshi,
>>>>>
>>>>> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
>>>>> <h-inoue@dream.email.ne.jp> wrote:
>>>>>>
>>>>>> Hi Igor,
>>>>>>
>>>>>>
>>>>>> On 2017/08/02 12:40, Igor Korot wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>     Hi,
>>>>>>>> I have a very weird situation.
>>>>>>>>
>>>>>>>> I am trying to develop an application which will connect to the
>>>>>>>> PostgreSQL
>>>>>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>>>>>
>>>>>>>> Environment:
>>>>>>>> MacOSX 10.8.
>>>>>>>> Xcode 4 (latest available for 10.8)
>>>>>>>> PostgreSQL 9.1 - original from OSX.
>>>>>>>> Application is compiled with the UNICODE
>>>>>>>> iODBC DM for OSX 10.8.
>>>>>>>> Both application and DB server running on the same machine.
>>>>>>>> The database I'm using was created with the default parameters (just
>>>>>>>> CREATE DATABASE draft).
>>>>>>>>
>>>>>>>> When I connect with the ODBC driver, the application works as
>>>>>>>> expected
>>>>>>>> - no errors or problems.
>>>>>>>>
>>>>>>>> However when I tried to run the application with libpq I received an
>>>>>>>> error. The error is:
>>>>>>>>
>>>>>>>> [quote]
>>>>>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>>>>>> [/quote]
>>>>>>>>
>>>>>>>> The application is written in C++ (not ObjC++).
>>>>>>>> The table I am trying to query contains the field with the word
>>>>>>>> "abc+<ALT225>", which inside the database is translated as
>>>>>>>> "abc\xc3\x9f".
>>>>>>>>
>>>>>>>> What could be the reason for such behavior?
>>>>>>>>
>>>>>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>>>>>> Or there something else?
>>>>>>>
>>>>>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>>>>>
>>>>>>> [code]
>>>>>>> [postgre]
>>>>>>> Driver               = PostgreSQL Unicode Driver
>>>>>>> host                 = localhost
>>>>>>> ServerName           = localhost
>>>>>>> SERVER               = localhost
>>>>>>> PORT                 = 5432
>>>>>>> UserName             = postgres
>>>>>>> Password             = <my_passwd>
>>>>>>> Database             = draft
>>>>>>> UseServerSidePrepare = 1
>>>>>>> [/code]
>>>>>>>
>>>>>>> As you can see there is no special encoding settings. So it uses one
>>>>>>> that is set by default.
>>>>>>> But using ODBC API works and trying libpq - doesn't.
>>>>>>>
>>>>>>> Could someone please help?
>>>>>>
>>>>>>
>>>>>> What is a query which causes a failure?
>>>>>> What is a result of locale command?
>>>>>
>>>>> Sorry for the noise.
>>>>> I think I found the culprit and in the process of fixing it.
>>>>>
>>>>> I will let you know how it goes.
>>>>
>>>> Apparently I didn't fix it yet.
>>>>
>>>> So here is information requested:
>>>>
>>>> MyMac:dbhandler igorkorot$ locale
>>>> LANG="en_US.UTF-8"
>>>> LC_COLLATE="en_US.UTF-8"
>>>> LC_CTYPE="en_US.UTF-8"
>>>> LC_MESSAGES="en_US.UTF-8"
>>>> LC_MONETARY="en_US.UTF-8"
>>>> LC_NUMERIC="en_US.UTF-8"
>>>> LC_TIME="en_US.UTF-8"
>>>> LC_ALL=
>>>>
>>>> And the failing query is:
>>>>
>>>> SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
>>>> $2 AND \"abc_cnam\" = $3;
>>>>
>>>> First parameter is "public.abc\xc3\x9f", second - "public" and third -
>>>> "id".
>>>>
>>>> The error is "Invalid key sequence for UTF-8".
>>>
>>>
>>> Do you call setlocale(LC_ALL, "") first?
>>
>> No I do not.
>> Is it what ODBC driver does?
>
>
> Yes.
> Unless setlocale() is called, "C" locale is selected.

I put in following:

[code]
class PostgreSQLDB
{
public:
    PostgreSQLDB();
private:
    char m_oldLocale[32], *m_currentlocale;
};

PostgreSQLDB::PostgreSQLDB()
{
    strcpy( m_oldLocale, setlocale( LC_ALL, NULL ) );
    m_currentlocale = setlocale( LC_ALL, "" );
}
[/code]

Looking under the debugger I see that:

m_oldLocale = "C/UTF-8/C/C/C/C"
m_currentLocale = "C"

So it looks like on Mac OSX it uses "C" locale when there is "" passed in.

But then I don't understand why ODBC app works....

Any idea?

Thank you.

>
>
> regards,
> Hiroshi Inoue
>
>> Thank you.
>>
>> BTW, with ODBC the table name is successfully converted to be
>> "abc<ALT+225>"
>> (it's a greek letter "beta" or a symbol for a German "ss" sound.- ß).
>>
>>
>>> regards,
>>> Hiroshi Inoue
>>>>
>>>>
>>>> Thank you.
>>>>
>>>>> Thank you.
>>>>>
>>>>>> regards,
>>>>>> Hiroshi Inoue
>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>>
>>>>>>>> Thank you.
>
>

Re: [ODBC] ODBC application works while libpq fails

From

"Inoue, Hiroshi"

Date:

07 August 2017, 08:30:18

Hi Igor,

On 2017/08/05 13:18, Igor Korot wrote:
> Hiroshi,
>
> On Fri, Aug 4, 2017 at 7:22 PM, Inoue, Hiroshi
> <h-inoue@dream.email.ne.jp> wrote:
>> Hi Igor,
>>
>>
>> On 2017/08/04 22:22, Igor Korot wrote:
>>> HI, Hiroshi,
>>>
>>> On Fri, Aug 4, 2017 at 9:17 AM, Inoue, Hiroshi
>>> <h-inoue@dream.email.ne.jp> wrote:
>>>> Hi Igor,
>>>>
>>>>
>>>> On 2017/08/04 20:59, Igor Korot wrote:
>>>>> Hi, Hiroshi,
>>>>>
>>>>> On Thu, Aug 3, 2017 at 12:06 AM, Igor Korot <ikorot01@gmail.com> wrote:
>>>>>> Hi, Hiroshi,
>>>>>>
>>>>>> On Wed, Aug 2, 2017 at 12:32 AM, Inoue, Hiroshi
>>>>>> <h-inoue@dream.email.ne.jp> wrote:
>>>>>>> Hi Igor,
>>>>>>>
>>>>>>>
>>>>>>> On 2017/08/02 12:40, Igor Korot wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On Mon, Jul 31, 2017 at 9:05 AM, Igor Korot <ikorot01@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>      Hi,
>>>>>>>>> I have a very weird situation.
>>>>>>>>>
>>>>>>>>> I am trying to develop an application which will connect to the
>>>>>>>>> PostgreSQL
>>>>>>>>> server thru 2 different methods: thru the ODBC and using libpq.
>>>>>>>>>
>>>>>>>>> Environment:
>>>>>>>>> MacOSX 10.8.
>>>>>>>>> Xcode 4 (latest available for 10.8)
>>>>>>>>> PostgreSQL 9.1 - original from OSX.
>>>>>>>>> Application is compiled with the UNICODE
>>>>>>>>> iODBC DM for OSX 10.8.
>>>>>>>>> Both application and DB server running on the same machine.
>>>>>>>>> The database I'm using was created with the default parameters (just
>>>>>>>>> CREATE DATABASE draft).
>>>>>>>>>
>>>>>>>>> When I connect with the ODBC driver, the application works as
>>>>>>>>> expected
>>>>>>>>> - no errors or problems.
>>>>>>>>>
>>>>>>>>> However when I tried to run the application with libpq I received an
>>>>>>>>> error. The error is:
>>>>>>>>>
>>>>>>>>> [quote]
>>>>>>>>> Invalid byte sequence for encoding UTF-8 0xc3\n
>>>>>>>>> [/quote]
>>>>>>>>>
>>>>>>>>> The application is written in C++ (not ObjC++).
>>>>>>>>> The table I am trying to query contains the field with the word
>>>>>>>>> "abc+<ALT225>", which inside the database is translated as
>>>>>>>>> "abc\xc3\x9f".
>>>>>>>>>
>>>>>>>>> What could be the reason for such behavior?
>>>>>>>>>
>>>>>>>>> Is ODBC driver sets the client encoding to some uniform value?
>>>>>>>>> Or there something else?
>>>>>>>> Here is the content of my .odbc.ini for PostgreSQL on Mac:
>>>>>>>>
>>>>>>>> [code]
>>>>>>>> [postgre]
>>>>>>>> Driver               = PostgreSQL Unicode Driver
>>>>>>>> host                 = localhost
>>>>>>>> ServerName           = localhost
>>>>>>>> SERVER               = localhost
>>>>>>>> PORT                 = 5432
>>>>>>>> UserName             = postgres
>>>>>>>> Password             = <my_passwd>
>>>>>>>> Database             = draft
>>>>>>>> UseServerSidePrepare = 1
>>>>>>>> [/code]
>>>>>>>>
>>>>>>>> As you can see there is no special encoding settings. So it uses one
>>>>>>>> that is set by default.
>>>>>>>> But using ODBC API works and trying libpq - doesn't.
>>>>>>>>
>>>>>>>> Could someone please help?
>>>>>>>
>>>>>>> What is a query which causes a failure?
>>>>>>> What is a result of locale command?
>>>>>> Sorry for the noise.
>>>>>> I think I found the culprit and in the process of fixing it.
>>>>>>
>>>>>> I will let you know how it goes.
>>>>> Apparently I didn't fix it yet.
>>>>>
>>>>> So here is information requested:
>>>>>
>>>>> MyMac:dbhandler igorkorot$ locale
>>>>> LANG="en_US.UTF-8"
>>>>> LC_COLLATE="en_US.UTF-8"
>>>>> LC_CTYPE="en_US.UTF-8"
>>>>> LC_MESSAGES="en_US.UTF-8"
>>>>> LC_MONETARY="en_US.UTF-8"
>>>>> LC_NUMERIC="en_US.UTF-8"
>>>>> LC_TIME="en_US.UTF-8"
>>>>> LC_ALL=
>>>>>
>>>>> And the failing query is:
>>>>>
>>>>> SELECT * FROM \"abcatcol\" WHERE \"abc_tnam\" = $1 AND \"abc_ownr\" =
>>>>> $2 AND \"abc_cnam\" = $3;
>>>>>
>>>>> First parameter is "public.abc\xc3\x9f", second - "public" and third -
>>>>> "id".

Could you send me directly the code around the query?

regards,
Hiroshi Inoue

>>>>>
>>>>> The error is "Invalid key sequence for UTF-8".
>>>>
>>>> Do you call setlocale(LC_ALL, "") first?
>>> No I do not.
>>> Is it what ODBC driver does?
>>
>> Yes.
>> Unless setlocale() is called, "C" locale is selected.
> I put in following:
>
> [code]
> class PostgreSQLDB
> {
> public:
>      PostgreSQLDB();
> private:
>      char m_oldLocale[32], *m_currentlocale;
> };
>
> PostgreSQLDB::PostgreSQLDB()
> {
>      strcpy( m_oldLocale, setlocale( LC_ALL, NULL ) );
>      m_currentlocale = setlocale( LC_ALL, "" );
> }
> [/code]
>
> Looking under the debugger I see that:
>
> m_oldLocale = "C/UTF-8/C/C/C/C"
> m_currentLocale = "C"
>
> So it looks like on Mac OSX it uses "C" locale when there is "" passed in.
>
> But then I don't understand why ODBC app works....
>
> Any idea?
>
> Thank you.
>
>>
>> regards,
>> Hiroshi Inoue
>>
>>> Thank you.
>>>
>>> BTW, with ODBC the table name is successfully converted to be
>>> "abc<ALT+225>"
>>> (it's a greek letter "beta" or a symbol for a German "ss" sound.- ß).
>>>
>>>
>>>> regards,
>>>> Hiroshi Inoue
>>>>>
>>>>> Thank you.
>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>>> regards,
>>>>>>> Hiroshi Inoue
>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thank you.