> buffer lenghth). So my proposal is, if prepare_buffer() detects> possible inconsistency between buffer encoding and
fileencoding, warn> user.
I agree with that.
On 2012/11/30 12:52, Tatsuo Ishii Wrote:
>>> 1. some especial character
>>> (my sql file contains japanese comment "-- コメント" . It can cause
>>> psql crash.)
>>> 2. PGCLIENTENCODING is SJIS
>>> 3. the encoding of input sql file is UTF-8
>
> Actually the problem can occur even when importing following 3 byte
> UTF8 input file:
>
> ト
>
> (in hexa, 0xe3, 0x83, 0x88)
>
> In this paticular case, psql decides that the total character length is
> 5, not 3. Because it just looks at the each first byte by calling PQmblen:
>
> 0xe3 -> 1 bytes in SJIS
> 0x83 -> 2 bytes in SJIS
> 0x88 -> 2 bytes in SJIS
> total: 5 bytes
>
> which is apparently wrong and causes subsequent segfault. Note that it
> is possible that "input file > psql decision" case as well if client
> encoding is different from file encoding, which will not be good too.
> I think we should detect the cases as much as possible and warn users,
> rather than silently ignore that fact client encoding != file
> encoding. I don't think we can detect it in a reliable way, but at
> least we could check the cases above(sum of PQmblen is not equale to
> buffer lenghth). So my proposal is, if prepare_buffer() detects
> possible inconsistency between buffer encoding and file encoding, warn
> user.
>
> [t-ishii@localhost psql]$ PGCLIENTENCODING=SJIS psql postgres
> Pager usage is off.
> psql (9.3devel)
> Type "help" for help.
>
> postgres=# \i ~/sql
> CREATE DATABASE
> You are now connected to database "mydb" as user "t-ishii".
> CREATE SCHEMA
> psql:/home/t-ishii/sql:7: warning: possible conflict between client encoding SJIS and input file encoding
> CREATE TABLE
>
> Comments?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
>
