Hi,
On 2025-02-15 12:35:45 -0800, Jeff Davis wrote:
> I am not suggesting a change, but there's a minor point about the
> behavior of the replacement that I'd like to highlight:
>
> Unicode discusses a choice[1]: "An ill-formed subsequence consisting of
> more than one code unit could be treated as a single error or as
> multiple errors."
>
> The patch implements the latter. Escaping:
> <7A F0 80 80 41 7A>
> results in:
> <7A C0 20 C0 20 C0 20 41 7A>
>
> The Unicode standard suggests[2] that the former approach may provide
> more consistency in how it's done, but that doesn't seem important or
> relevant for our purposes. I'd favor whichever approach results in
> simpler code.
It seems completely infeasible to me to to implement the "single error"
approach in a minor version. It'd afaict require non-trivial new
infrastructure. We can't just consume up to the next byte without a high bit,
because some encodings have subsequent bytes that are not guaranteed to have a
high bit set.
Greetings,
Andres Freund
