Thread: COPY FROM encoding error

Hi list !

I am trying to migrate a database from MSSQL to PostgreSQL.
I created the table in PostgreSQL, and I exported the data
as CSV from MSSQL's Enterprise Manager.

In the "Export Data" Wizard, I chose the option to export as
UNICODE.

In PostgreSQL, I try to load the data using :

COPY mytable (col1, col2, col3)
FROM 'mytable.csv' CSV;

But I receive this error after some time :
ERROR: invalid byte sequence for encoding "UTF8": 0xff
État SQL :22021
Astuce : This error can also happen if the byte sequence
does not match the encoding expected by the server, which is
controlled by "client_encoding".
Contexte : COPY mytable, line 592680

I think that the encoding is OK because more than 500.000
lines are copied without problem, so there might be just one
problematic character here.

How can I solve this problem ?

Thanks a lot !
--
Arnaud

On Nov 21, 2006, at 4:20 AM, Arnaud Lesauvage wrote:
> ERROR: invalid byte sequence for encoding "UTF8": 0xff
> État SQL :22021
> Astuce : This error can also happen if the byte sequence does not
> match the encoding expected by the server, which is controlled by
> "client_encoding".
> Contexte : COPY mytable, line 592680
>
> I think that the encoding is OK because more than 500.000 lines are
> copied without problem, so there might be just one problematic
> character here.
>
> How can I solve this problem ?

You need to fix the bad character. You can do this manually, or
search the archives for "UTF8 invalid iconv" for another solution.
--
Jim Nasby                               jim.nasby@enterprisedb.com
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)