Re: proposal: UTF8 to_ascii function - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: proposal: UTF8 to_ascii function
Date
Msg-id 48A0303B.6010504@dunslane.net
Whole thread Raw
In response to proposal: UTF8 to_ascii function  ("Pavel Stehule" <pavel.stehule@gmail.com>)
Responses Re: proposal: UTF8 to_ascii function  ("Pavel Stehule" <pavel.stehule@gmail.com>)
Re: proposal: UTF8 to_ascii function  ("Pavel Stehule" <pavel.stehule@gmail.com>)
List pgsql-hackers

Pavel Stehule wrote:
> Hello,
>
> combination functions to_ascii and convert_to is broken now. Problem
> is in convert_to function. It doesn't support 8bit output encoding.
>
> Current workaround:
>
> CREATE FUNCTION to_ascii(bytea, name)
> RETURNS text AS 'to_ascii_encname' LANGUAGE internal;
>
> SELECT to_ascii(convert_to('Příliš žlutý kůň', 'latin2'),'latin2');
>
> I don't expect column collate for 8.4, so we need to have workable
> to_ascii function.
>
> I propose function to_ascii(text, name) that internally convert text
> from utf8 encoding when it's necessary.cheers
>
>
>   

convert_to is not broken. It returns a bytea, and it is up to you to 
de-escape it if you get the text representation.

We are surely not going to go back to a situation where we have 
functions returning text in any encoding other than the database 
encoding. That becomes a vehicle for storing wrongly encoded data in the 
database, and we have just gone through the exercise of plugging those 
holes. I privately predicted when we did this work that it might 
motivate people who had been abusing convert_to to get proper support 
for multiple encodings done. That is the right way to go, not re-opening 
holes we have just very deliberately plugged.



cheers

andrew


pgsql-hackers by date:

Previous
From: "Heikki Linnakangas"
Date:
Subject: Re: Proposal: PageLayout footprint
Next
From: Zdenek Kotala
Date:
Subject: Re: Proposal: PageLayout footprint