Re: EOL characters and multibyte encodings - Mailing list pgsql-hackers

From William ZHANG
Subject Re: EOL characters and multibyte encodings
Date
Msg-id f5g1gh$1b7$1@news.hub.org
Whole thread Raw
In response to EOL characters and multibyte encodings  (Joe Conway <mail@joeconway.com>)
Responses Re: EOL characters and multibyte encodings  (Andrew Dunstan <andrew@dunslane.net>)
List pgsql-hackers
"Joe Conway" <mail@joeconway.com>
> Tom Lane wrote:
>> Joe Conway <mail@joeconway.com> writes:
>>> My first thought on fixing this issue was to simply replace all 
>>> instances of '\r' in pg_proc.prosrc with '\n' prior to sending it to the 
>>> R parser. As far as I know, any instances of '\r' embedded in a 
>>> syntactically valid R statement must be escaped (i.e. literally the 
>>> characters "\" and "r"), so that should not be a problem. But I am 
>>> concerned about how this potentially plays against multibyte characters. 
>>> Is it safe to do this, or do I need to use a mb-aware replace algorithm?
>>
>> It's safe, because you'll be dealing with prosrc inside the backend,
>> therefore using a backend-legal encoding, and those don't have any ASCII
>> aliasing problems (all bytes of an MB character must have high bit set).

The lower byte of some characters in BIG5, GBK, GB18030 may be less than
0x7F and don't have the high bit set. Fortunately, they don't use 0x0D and
0x0A (CR and LF).

Regards,
William ZHANG

> Great -- I wasn't sure about that.
>




pgsql-hackers by date:

Previous
From: Zdenek Kotala
Date:
Subject: Re: What does Page Layout version mean? (Was: Re: Reducing NUMERIC size for 8.3)
Next
From: "Simon Riggs"
Date:
Subject: Re: Worries about delayed-commit semantics