Re: Bug in UTF8-Validation Code? - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Bug in UTF8-Validation Code?
Date
Msg-id 9805.1175619998@sss.pgh.pa.us
Whole thread Raw
In response to Re: Bug in UTF8-Validation Code?  (Mark Dilger <pgsql@markdilger.com>)
Responses Re: Bug in UTF8-Validation Code?  (Martijn van Oosterhout <kleptog@svana.org>)
List pgsql-hackers
Mark Dilger <pgsql@markdilger.com> writes:
> Martijn van Oosterhout wrote:
>> Just about every multibyte encoding other than Unicode has the problem
>> of not distinguishing between the code point and the encoding of it.

> Thanks for the feedback.  Would you say that the way I implemented things in the 
> example code would be correct for multibyte non Unicode encodings?

I think it's probably defensible for non-Unicode encodings.  To do
otherwise would require (a) figuring out what the equivalent concept to
"code point" is for each encoding, and (b) having a separate code path
for each encoding to perform the mapping.  It's not clear that there
even is an answer to (a), and (b) seems like more work than chr() is
worth.  But we know what the right way is for Unicode, so we should
special case that one.

Note the points made that in all cases ascii() and chr() should be
inverses, and that you shouldn't just fall back to the old behavior
in SQL_ASCII encoding.  (My vote for SQL_ASCII would be to reject
values > 255.)
        regards, tom lane


pgsql-hackers by date:

Previous
From: Josh Berkus
Date:
Subject: Re: Implicit casts to text
Next
From: Tom Lane
Date:
Subject: Re: Implicit casts to text