Re: Need magic to clean strings from unconvertible UTF8 - Mailing list pgsql-general

On 11/06/10 9:35 PM, Andreas wrote:
> Hi,
>
> somehow there have unconvertible characters sneaked into my DB.
> Very probaply they came in via Imports from MS-Access.
>
> Access doesn't complain but when I try to export stuff with pgAdmin to
> csv I get an error that some char is not representable in the local
> charset.
>
> I can find the problematic rows.
> How could I delete every char in a string that can't be converted to
> WIN1252?


One idea that comes to my mind....  issue a

     SET CLIENT_ENCODING 'C';

then find and fix any problems with SQL.     The C aka Posix encoding
lets you directly manipulate the characters as binary.

or set the client_encoding to whatever the database encoding is, and
find the characters that you know aren't compatible with WIN1252 and
change them