Thread: problem: Postgresql and Unicode

problem: Postgresql and Unicode

From
"Nathalie Boulos"
Date:
Hi everyone,

I'm using postgresql-7.2.2-1 under RedHat linux 8.0.

I created a unicode DB and trying to insert arabic text through a php script
on the web. I'm getting the following error:
Warning: pg_exec() query failed: ERROR: Invalid UNICODE character sequence
found (0xe40000)

the client is a IE browser on a win2000 machine, with arabic support.
(I'm forcing client encoding to unicode before i insert data in the DB by
adding the following line in the PHP script:
pg_exec($db,"set client_encoding='unicode'");       )

Any ideas?

I'd greatly appreciate any help and thanks in advance

Regards
Nathalie


_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE*
http://join.msn.com/?page=features/junkmail


Re: problem: Postgresql and Unicode

From
Antti Haapala
Date:
On Fri, 21 Feb 2003, Nathalie Boulos wrote:

> I created a unicode DB and trying to insert arabic text through a php
> script on the web. I'm getting the following error: Warning: pg_exec()
> query failed: ERROR: Invalid UNICODE character sequence found (0xe40000)
>
> the client is a IE browser on a win2000 machine, with arabic support.
> (I'm forcing client encoding to unicode before i insert data in the DB by
> adding the following line in the PHP script:
> pg_exec($db,"set client_encoding='unicode'");       )

And in what encoding does PHP send data to backend? It should be in
UTF-8...

    perl -e 'print "\xe4\x00\x00"' | iconv -f utf-8 -t utf-8
prints
    iconv: illegal input sequence at position 0

iconv doesn't seem to like that either.

--
Antti Haapala


Re: problem: Postgresql and Unicode

From
Artur Pietruk
Date:
On Fri, Feb 21, 2003 at 11:01:31AM +0200, Nathalie Boulos wrote:
> Hi everyone,

    Hi!

> I'm using postgresql-7.2.2-1 under RedHat linux 8.0.
>
> I created a unicode DB and trying to insert arabic text through a php
> script on the web. I'm getting the following error:
> Warning: pg_exec() query failed: ERROR: Invalid UNICODE character sequence
> found (0xe40000)
>
> the client is a IE browser on a win2000 machine, with arabic support.
> (I'm forcing client encoding to unicode before i insert data in the DB by
> adding the following line in the PHP script:
> pg_exec($db,"set client_encoding='unicode'");       )

    Could you show us how does your PHP script looks like? Setting
client encoding on the postgres side, can be not enough. Did you set
encoding of your PHP page to UTF, using content-type meta tag? Make sure,
that data you want to insert, is in fact unicode. I'm just
wild-guessing, I have never created unicode DB in postgres, but I was
fighting with enocding issues here and there.

    Best regards,
--
--- Artur Pietruk, arturp@plukwa.net

Re: problem: Postgresql and Unicode

From
"Nathalie Boulos"
Date:

Hi Arthur ,

Thank you very much for your reply.
It was indeed a PHP problem. I just set the default_charset variable in
php.ini to utf-8 and it worked.

But now I'm facing another problem:
How do I convert existing data in an SQL_ASCII database to unicode?

here's my problem:
I have a SQL_ASCII postgres db containing data.
I've updated the pg_database table and set the encoding  to UNICODE for this
database.
I've changed the php.ini charset variable to utf-8.
When i add new rows to the table, I can read and write successfully in
unicode.
But if i want to see the old existing data, i have problems visualizing the
special characters (like french lettres with accents).
So i guess i have to convert the old data.

Has anyone tried this before?

Thanks for your help.

Regards
Nat


>
>On Fri, Feb 21, 2003 at 11:01:31AM +0200, Nathalie Boulos wrote:
> > Hi everyone,
>
>    Hi!
>
> > I'm using postgresql-7.2.2-1 under RedHat linux 8.0.
> >
> > I created a unicode DB and trying to insert arabic text through a php
> > script on the web. I'm getting the following error:
> > Warning: pg_exec() query failed: ERROR: Invalid UNICODE character
>sequence
> > found (0xe40000)
> >
> > the client is a IE browser on a win2000 machine, with arabic support.
> > (I'm forcing client encoding to unicode before i insert data in the DB
>by
> > adding the following line in the PHP script:
> > pg_exec($db,"set client_encoding='unicode'");       )
>
>    Could you show us how does your PHP script looks like? Setting
>client encoding on the postgres side, can be not enough. Did you set
>encoding of your PHP page to UTF, using content-type meta tag? Make sure,
>that data you want to insert, is in fact unicode. I'm just
>wild-guessing, I have never created unicode DB in postgres, but I was
>fighting with enocding issues here and there.
>
>    Best regards,
>--
>--- Artur Pietruk, arturp@plukwa.net
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: Have you searched our list archives?
>
>http://archives.postgresql.org


_________________________________________________________________
Tired of spam? Get advanced junk mail protection with MSN 8.
http://join.msn.com/?page=features/junkmail


Re: problem: Postgresql and Unicode

From
Björn Metzdorf
Date:
> But now I'm facing another problem:
> How do I convert existing data in an SQL_ASCII database to unicode?
>
> Has anyone tried this before?

Yes, that is no problem. Have a look at
http://www.postgresql.org/docs/view.php?version=7.3&idoc=1&file=multibyte.ht
ml#AEN21920

short: dump sql_ascii, create unicode db, insert the dump with client
encoding set to latin1 (or whatever).

Regards,
Bjoern