Re: UTF16 surrogate pairs in UTF8 encoding - Mailing list pgsql-hackers

From Florian Weimer
Subject Re: UTF16 surrogate pairs in UTF8 encoding
Date
Msg-id 82vd71kekk.fsf@mid.bfk.de
Whole thread Raw
In response to UTF16 surrogate pairs in UTF8 encoding  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
* Tom Lane:

> I just noticed that we are now advertising the ability to insert UTF16
> surrogate pairs in strings and identifiers (see section 4.1.2.2 in
> current docs, in particular).  Is this really wise?  I thought that
> surrogate pairs were specifically prohibited in UTF8 strings, because
> of the security hazards implicit in having more than one way to
> represent the same code point.

There is relatively little risk because surrogate pairs cannot encode
characters in the BMP, and presumably, most of the critical characters
are located there.

However, if this is converted to regular UTF-8, I really question the
sense of this.  Usually, people want CESU-8 to preserve ordering
between languages such as C# and Java and their database, and
conversion destroys this property.

--
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: pg_archivecleanup debug message consistency
Next
From: Magnus Hagander
Date:
Subject: Re: git: uh-oh