Re: Pg 7.4 to 8.1 UTF problems - Mailing list pgsql-admin

From Tom Lane
Subject Re: Pg 7.4 to 8.1 UTF problems
Date
Msg-id 8168.1139674477@sss.pgh.pa.us
Whole thread Raw
In response to Pg 7.4 to 8.1 UTF problems  (Mario Splivalo <msplival@jagor.srce.hr>)
List pgsql-admin
Mario Splivalo <msplival@jagor.srce.hr> writes:
> psql:the_database.psql:86: ERROR:  invalid UTF-8 byte sequence detected
> near byte 0x8d
> CONTEXT:  COPY _netsms, line 1367, column text: "Padamo u pozu 69.uz
> \uffffmas mog macana u usta.a ja tvoju ljepu picu"

7.4's checking for valid UTF8 code sequences had some bugs, causing it
to accept data that is not valid UTF8.  8.1 has tightened that up.

> From here it is visible that the troubled charachter has has ASCII HEX
> code 8d, which is regular ascii charachter.

It is not ASCII, and it is not legal UTF8 either, at least not without
another byte >= 0x80 after it.

You need to decide whether this is bad data (and if so fix it), or
whether you misdetermined what the encoding of your data is (and if
so, change to the correct encoding declaration).

            regards, tom lane

pgsql-admin by date:

Previous
From: "Magnus Hagander"
Date:
Subject: Re: Could not start the PostgreSQL Database Server [version # inserted here] service on local computer
Next
From: Tom Lane
Date:
Subject: Ancient messages regurgitated into pgsql-admin, again