Re: MSSQL to PostgreSQL : Encoding problem - Mailing list pgsql-general

From Alvaro Herrera
Subject Re: MSSQL to PostgreSQL : Encoding problem
Date
Msg-id 20061122170046.GE6804@alvh.no-ip.org
Whole thread Raw
In response to Re: MSSQL to PostgreSQL : Encoding problem  (Arnaud Lesauvage <thewild@freesurf.fr>)
Responses Re: MSSQL to PostgreSQL : Encoding problem  (Arnaud Lesauvage <thewild@freesurf.fr>)
List pgsql-general
Arnaud Lesauvage wrote:
> Alvaro Herrera a écrit :
> >Arnaud Lesauvage wrote:
> >
> >>mydb=# SET client_encoding TO LATIN9;
> >>SET
> >>mydb=# COPY statistiques.detailrecherche (log_gid,
> >>champrecherche, valeurrecherche) FROM
> >>'E:\\Production\\Temp\\detailrecherche_ansi.csv' CSV;
> >>ERROR:  invalid byte sequence for encoding "LATIN9": 0x00
> >>HINT:  This error can also happen if the byte sequence does
> >>not match the encoding expected by the server, which is
> >>controlled by "client_encoding".
> >
> >Huh, why do you have a "0x00" byte in there?  That's certainly not
> >Latin9 (nor UTF8 as far as I know).
> >
> >Is the file actually Latin-something or did you convert it to something
> >else at some point?
>
> This is the file generated by DTS with "ANSI" encoding. It
> was not altered in any way after that !
> The doc states that ANSI exports with the local codepage
> (which is Win1252). That's all I know. :(

I thought Win1252 was supposed to be almost the same as Latin1.  While
I'd expect certain differences, I wouldn't expect it to use 0x00 as
data!

Maybe you could have DTS export Unicode, which would presumably be
UTF-16, then recode that to something else (possibly UTF-8) with GNU
iconv.

FWIW, I think the preferred way to set the client encoding on psql is
\encoding.  I'm not sure if it does anything different from the SET
command though.

--
Alvaro Herrera       Valdivia, Chile   ICBM: S 39º 49' 18.1", W 73º 13' 56.4"
"In Europe they call me Niklaus Wirth; in the US they call me Nickel's worth.
 That's because in Europe they call me by name, and in the US by value!"

pgsql-general by date:

Previous
From: Tony Caduto
Date:
Subject: Re: Data transfer between databases over the Internet
Next
From: Tom Lane
Date:
Subject: Re: Buffer overflow in psql