Jeff Davis <pgsql@j-davis.com> writes:
> We should probably just check that it's plain ASCII.
That was about the same conclusion I came to.  More or less
    c = *v->now++;
-    if (!iscalnum(c))
+    if (!isascii(c) || !iscalnum(c))
        RETV(PLAIN, c);
although I'm not sure if it's a good idea to apply isascii()
to something that's wider than char.  It might be advisable,
if ugly, to write
+    if (c >= 0x100 || !iscalnum(c))
I'm also inclined to remove that "assert(iscalpha(c))" in
the switch.  That's not checking sanity of our own code,
just consistency of the <wctype.h> functions; and as this
episode shows it's more trouble than it's worth.
            regards, tom lane