On Wed, Nov 20, 2024 at 10:54:50AM -0500, Tom Lane wrote:
> Nathan Bossart <nathandbossart@gmail.com> writes:
>> That's good to know. If we can assume that 1) all bytes of a multibyte
>> character have the high bit set and 2) all multibyte characters actually
>> require multiple bytes, then there are just a handful of cases that require
>> multiple lookups, and we can restrict even those to some extent, too.
>
> I'm failing to parse your (2). Either that's content-free or you're
> thinking something that probably isn't true. There are encodings
> (mostly the LATINn series) that have high-bit-set characters that
> only occupy one byte. So I don't think we can take any shortcuts
> compared to the strip-one-byte-at-a-time approach.
I'm probably missing something here, sorry.
Upthread, you mentioned that we could bypass multiple lookups unless both
the NAMEDATALEN-1'th and NAMEDATALEN-2'th bytes are non-ASCII. But if
there are encodings with the high bit set that don't require multiple bytes
per character, then how can we do that? For example, let's say the
initially-truncated identifier ends with an ASCII byte followed by a
non-ASCII byte. That last byte might be a LATIN1 character, or it could be
the beginning of a character that requires multiple bytes, so we need to
lookup both the initially truncated string as well as the string with one
extra byte truncated, right?
--
nathan
