invalid byte sequence for encoding "UTF8" - Mailing list pgsql-general

From Fuzzygoth
Subject invalid byte sequence for encoding "UTF8"
Date
Msg-id 1174475827.299904.35650@l75g2000hse.googlegroups.com
Whole thread Raw
Responses Re: invalid byte sequence for encoding "UTF8"  (Alan Hodgson <ahodgson@simkin.ca>)
List pgsql-general
Hi,

I am trying currently trying to setup our new database sever, we have
upgraded
to PostgreSQL 8.1.8. When I try to restore the backup (which is stored
as a set
of SQL statements that my restore script feeds into PSQL to execute)
it returns
the following error.

psql:/mnt/tmp/app/application_data.sql:97425: ERROR:  invalid byte
sequence for encoding "UTF8": 0xff

HINT:  This error can also happen if the byte sequence does not match
the encoding expected by the server, which is controlled by
"client_encoding".

along other byte sequences eg: 0xa1, 0xac, the two remaining schemas
are
roughly 22GB and 66GB in size and is read into postgres from flat
cobol
datafiles.

our data has progressed as displayed below
PostgreSQL 7.?.? Stored in SQL-ASCII (Old configuration)
PostgreSQL 8.1.3 Stored in UTF8 (current conguration)
PostgreSQL 8.1.8 Stored in UTF8 (our future configuration)

The encoding type set on the server was changed to UTF8 from SQL-ASCII
after
we moved to version 8.1.3 for purposes of globalisation.

I've searched the forums and found people with similar problems but
not much
on a way to remedy it. I did try using iconv which was suggested in a
thread
but it returned an error saying even the 22GB file was too large to
work on.

any help would be gratfully appreciated.

Many Thanks
David P


pgsql-general by date:

Previous
From: Benjamin Arai
Date:
Subject: multi terabyte fulltext searching
Next
From: "Tijnema !"
Date:
Subject: Re: [PHP] Re: phpPgAdmin - prior version available?