Re: Significance of Database Encoding - Mailing list pgsql-sql

From Rajesh Mallah
Subject Re: Significance of Database Encoding
Date
Msg-id 20050515183829.53021.qmail@web31011.mail.mud.yahoo.com
Whole thread Raw
In response to Significance of Database Encoding  (Rajesh Mallah <mallah_rajesh@yahoo.com>)
Responses Re: Significance of Database Encoding  (PFC <lists@boutiquenumerique.com>)
List pgsql-sql
--- PFC <lists@boutiquenumerique.com> wrote:
> 
> > +--------------------------------------------------------------------------------------+
> > | 私はガラス
> > +--------------------------------------------------------------------------------------+
> 
>     You say it displays correctly in xterm (ie. you didn't see these in your  
> xterm).
>     There are HTML/XML unicode character entities, probably generated by your  
> mailer from your Unicode cut'n'paste.

That is correct.

Now the question is how to convert from SQL_ASCII to UNICODE. 
Mailing lists suggests to run recode or iconv on the dump file
and restore. The problem is on running iconv with -f US-ASCII
the program aborted:

$ iconv -f US-ASCII -t UTF-8  < test.sql > out.sql
iconv: illegal input sequence at position 114500

Any ideas how the job can be accomplised reliably.

Also my database may contain data in multiple encodings
like WINDOWS-1251 and WINDOWS-1256 in various places
as data has been inserted by different peoples using
different sources and client software.


Regds
Rajesh Kumar Mallah.










>     Using SQL ASCII to store UTF8 encoded data will work, but postgres won't  
> know that it's manipulating multibyte characters, so for instance the  
> length of a string will be its Byte length instead of correctly counting  
> the characters, collation rules will be funky, etc. And substring() may  
> well cut in the middle of an UTF8 multibyte char which will then screw  
> your application side processing...
>     Apart from that, it'll work ;)
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


pgsql-sql by date:

Previous
From: PFC
Date:
Subject: Re: Significance of Database Encoding
Next
From: PFC
Date:
Subject: Re: Significance of Database Encoding