In src/backend/utils/mb/wchar.c, function ucs_wcwidth(), there is a list
of Unicode combining characters, so that those can be ignored for
computing the display length of a Unicode string. It seems to me that
that list is either outdated or plain incorrect.
For example, the list starts with
{0x0300, 0x034E}, {0x0360, 0x0362}, {0x0483, 0x0486},
Let's look at the characters around the first "gap":
(https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt)
034C;COMBINING ALMOST EQUAL TO ABOVE;Mn;230;NSM;;;;;N;;;;;
034D;COMBINING LEFT RIGHT ARROW BELOW;Mn;220;NSM;;;;;N;;;;;
034E;COMBINING UPWARDS ARROW BELOW;Mn;220;NSM;;;;;N;;;;;
034F;COMBINING GRAPHEME JOINER;Mn;0;NSM;;;;;N;;;;;
0350;COMBINING RIGHT ARROWHEAD ABOVE;Mn;230;NSM;;;;;N;;;;;
0351;COMBINING LEFT HALF RING ABOVE;Mn;230;NSM;;;;;N;;;;;
So these are all in the "Mn" category, so they should be treated all the
same here. Indeed, psql doesn't compute the width of some of them
correctly:
postgres=> select u&'|oo\034Coo|';
+----------+
| ?column? |
+----------+
| |oXoo| |
+----------+
postgres=> select u&'|oo\0350oo|';
+----------+
| ?column? |
+----------+
| |oXoo| |
+----------+
(I have replaced the combined character with X above so that the mail
client rendering doesn't add another layer of uncertainty to this issue.
The point is that the box is off in the second example.)
AFAICT, these Unicode definitions haven't changed since that list was
put in originally around 2006, so I wonder what's going on there.
I have written a script that recomputes that list from the current
Unicode data. Patch and script are attached. This makes those above
cases all render correctly. (This should eventually get better built
system integration.)
Thoughts?
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
