Re: invalid byte sequence for encoding "UTF8": 0xff - Mailing list pgsql-general

From Albe Laurenz
Subject Re: invalid byte sequence for encoding "UTF8": 0xff
Date
Msg-id D960CB61B694CF459DCFB4B0128514C229749A@exadv11.host.magwien.gv.at
Whole thread Raw
In response to Re: invalid byte sequence for encoding "UTF8": 0xff  ("Ashish Karalkar" <ashish.karalkar@info-spectrum.com>)
List pgsql-general
Ashish Karalkar wrote:
>>> I have a data script which runs fine from PgAdmin SQL
>>> Editor,but when I  run this  from command prompt I get
>>> following error:
>>>
>>> test=# \i /usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql
>>>
>>> psql:/usr/local/pgsql/qsweb1/QSWEB_100_4_Default_Data.sql:1:
>>> ERROR:  invalid byte sequence for encoding "UTF8": 0xff
>>>
>>>  version
>>> --------------------------------------------------------------
>>>  PostgreSQL 8.2.0 on i686-pc-linux-gnu, compiled by GCC gcc
>>> (GCC) 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)
>>
>> - What is the client operating system (where you run psql and
>>  PgAdmin III)?
>>
> Its Windows XP - PgAdmin III
> RHEL 3.4.3-9.EL4-psql (Server Machine)

So I get it that you run "psql" on Windows XP, right?

> PGCLIENTENCODING is not set and as per documantation I
> think by default it takes value of database i.e. UTF8
>
>> - What does the SQL command "show client_encoding;" return
>>  when you issue it in
>>  a) PgAdmin III
> UNICODE
>>  b) psql
> UTF8

Ok, I suspect that's your problem.
You created QSWEB_100_4_Default_Data.sql by using the "Save" dialog
in PgAdmin III on the Windows machine, right?

Then the file will probably be encoded in Windows-1252.

If your client_encoding is set to UTF8, psql will expect UTF-8
data in the SQL script and complain if it meets wrong ones.

Does the script work as expected when you change the client
encoding to WIN1252?

>> - Please create a file that contains only the first line
>>  of QSWEB_100_4_Default_Data.sql (I call it "l" in the following
>>  commands), run the following two (Linux) commands on it:
>>  a) od -t c l
>>  b) od -t x1 l
>>  and show us the output of both commands.
>
> [root@localhost qsweb]# od -t c test.sql
> 0000000   \   s   e   t       O   N   _   E   R   R   O   R   _   S
T
> 0000020   O   P
> 0000022
> [root@localhost qsweb]# od -t x1 test.sql
> 0000000 5c 73 65 74 20 4f 4e 5f 45 52 52 4f 52 5f 53 54
> 0000020 4f 50
> 0000022

That's weird, because psql complained about line 1.

Maybe you messed something up by extracting the first line.

Try the following:

- Use binary file transfer and transfer the SQL script to a Linux
machine.

- Run "od -t c -t x1" on the file

- Find the 0xff that psql complains about.

Maybe that helps to locate the problem.
0xff is an unusual Windows-1252 character as well...

Yours,
Laurenz Albe

pgsql-general by date:

Previous
From: "Ashish Karalkar"
Date:
Subject: Re: invalid byte sequence for encoding "UTF8": 0xff
Next
From: "Ashish Karalkar"
Date:
Subject: Re: invalid byte sequence for encoding "UTF8": 0xff