Re: Unicode normalization - Mailing list pgsql-general

From Scott Marlowe
Subject Re: Unicode normalization
Date
Msg-id dcc563d10909161601v1991b152q6bd2c2a829a511dd@mail.gmail.com
Whole thread Raw
In response to Re: Unicode normalization  (Sam Mason <sam@samason.me.uk>)
List pgsql-general
On Wed, Sep 16, 2009 at 4:42 PM, Sam Mason <sam@samason.me.uk> wrote:
> On Wed, Sep 16, 2009 at 09:35:02PM +0200, Andreas Kalsch wrote:
>> CREATE OR REPLACE FUNCTION test (str text)
>>  RETURNS text
>> AS $$
>>    import unicodedata
>>    return unicodedata.normalize('NFKD', str.decode('UTF-8'))
>> $$ LANGUAGE plpythonu;
>
> I'd guess you want that to be:
>
>  return unicodedata.normalize('NFKD', str.decode('UTF-8')).encode('UTF-8');
>
> If you're converting from a utf8 encoding, you probably need to go
> back again!  This could certainly be made easier though, PG knows what
> encoding its strings are stored in, why doesn't it work with unicode
> strings by default?

Isn't it python that's making the mistake here, not pg?

pgsql-general by date:

Previous
From: Nathan Widmyer
Date:
Subject: Re: Current state of XML capabilities in PostgreSQL?
Next
From: Andreas Kalsch
Date:
Subject: How to simplify unicode strings