Re: UTF8 with BOM support in psql - Mailing list pgsql-hackers

From Itagaki Takahiro
Subject Re: UTF8 with BOM support in psql
Date
Msg-id 20091117164023.1513.52131E4D@oss.ntt.co.jp
Whole thread Raw
In response to Re: UTF8 with BOM support in psql  (Peter Eisentraut <peter_e@gmx.net>)
Responses Re: UTF8 with BOM support in psql
List pgsql-hackers
Peter Eisentraut <peter_e@gmx.net> wrote:

> I think I could support using the presence of the BOM as a fall-back
> indicator of encoding in absence of any other declaration.

What is the difference the fall-back and <<set client encoding to UTF-8
if BOM found>> ? I read this discussion that we cannot accept any automatic
encoding detections (properly speaking, detection is ok, but automatic
assignment is not). We should not have any fall-back mechanism, no?

> Also, when the proposed patch to set the encoding from the locale
> appears, we need to make this logic more precise.

Encoding-from-locale feature will be useful, but the patch does *not*
set any encodings. The reason is same as above.

> Also, I'm not sure if we need this logic only when we send a query.  It
> might be better to do this in the lexer when we find a non-ASCII
> character and we don't have a client encoding != SQL_ASCII set yet.

Absolutely, but is it an indepedent issue from BOM? Multi-byte scripts
without encoding are always dangerous whether BOM is present or not.
I'd say we can always throw an error when we find queries that contain
multi-byte characters if no prior encoding declaration.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center




pgsql-hackers by date:

Previous
From: "Markus Wanner"
Date:
Subject: Re: write ahead logging in standby (streaming replication)
Next
From: "Joshua D. Drake"
Date:
Subject: Re: next CommitFest