Re: speed up verifying UTF-8 - Mailing list pgsql-hackers

From Vladimir Sitnikov
Subject Re: speed up verifying UTF-8
Date
Msg-id CAB=Je-Eqcuz2MxuA0QU-6qLDrG0bvRB+UBj7JoFekM1fxk_H_g@mail.gmail.com
Whole thread Raw
In response to Re: speed up verifying UTF-8  (John Naylor <john.naylor@enterprisedb.com>)
Responses Re: speed up verifying UTF-8  (John Naylor <john.naylor@enterprisedb.com>)
Re: speed up verifying UTF-8  (John Naylor <john.naylor@enterprisedb.com>)
List pgsql-hackers
Just wondering, do you have the code in a GitHub/Gitlab branch?

>+ utf8_advance(s, state, len);
>+
>+ /*
>+ * If we saw an error during the loop, let the caller handle it. We treat
>+ * all other states as success.
>+ */
>+ if (state == ERR)
>+ return 0;

Did you mean state = utf8_advance(s, state, len); there? (reassign state variable)

>I wanted to try different strides for the DFA

Does that (and "len >= 32" condition) mean the patch does not improve validation of the shorter strings (the ones less than 32 bytes)?
It would probably be nice to cover them as well (e.g. with 4 or 8-byte strides)

Vladimir

pgsql-hackers by date:

Previous
From: John Naylor
Date:
Subject: Re: speed up verifying UTF-8
Next
From: Ibrar Ahmed
Date:
Subject: Re: 2021-07 CF now in progress