Re: Unicode normalization - Mailing list pgsql-general

From Sam Mason
Subject Re: Unicode normalization
Date
Msg-id 20090916224258.GM5407@samason.me.uk
Whole thread Raw
In response to Re: Unicode normalization  (Andreas Kalsch <andreaskalsch@gmx.de>)
Responses Re: Unicode normalization  (Scott Marlowe <scott.marlowe@gmail.com>)
How to simplify unicode strings  (Andreas Kalsch <andreaskalsch@gmx.de>)
List pgsql-general
On Wed, Sep 16, 2009 at 09:35:02PM +0200, Andreas Kalsch wrote:
> CREATE OR REPLACE FUNCTION test (str text)
>  RETURNS text
> AS $$
>    import unicodedata
>    return unicodedata.normalize('NFKD', str.decode('UTF-8'))
> $$ LANGUAGE plpythonu;

I'd guess you want that to be:

  return unicodedata.normalize('NFKD', str.decode('UTF-8')).encode('UTF-8');

If you're converting from a utf8 encoding, you probably need to go
back again!  This could certainly be made easier though, PG knows what
encoding its strings are stored in, why doesn't it work with unicode
strings by default?

--
  Sam  http://samason.me.uk/

pgsql-general by date:

Previous
From: Scott Bailey
Date:
Subject: Re: Current state of XML capabilities in PostgreSQL?
Next
From: Nathan Widmyer
Date:
Subject: Re: Current state of XML capabilities in PostgreSQL?