Re: speed up verifying UTF-8 - Mailing list pgsql-hackers

From John Naylor
Subject Re: speed up verifying UTF-8
Date
Msg-id CAFBsxsHUcyxpRPYi22d0LOD+YQz4FsJo5POEWh47=zkwbypU=w@mail.gmail.com
Whole thread Raw
In response to Re: speed up verifying UTF-8  (John Naylor <john.naylor@enterprisedb.com>)
List pgsql-hackers


I wrote:

> On Thu, Jun 3, 2021 at 10:42 AM Greg Stark <stark@mit.edu> wrote:
> >

> > If
> > we're processing much more than 128 bits and happy to detect NUL
> > errors only at the end after wasting some work then you could hoist
> > that has_zero check entirely out of the loop (removing the branch
> > though it's probably a correctly predicted branch anyways).
> >
> > Do something like:
> >
> > zero_accumulator = zero_accumulator & next_chunk
> >
> > in the loop and then only at the very end check for zeros in that.
>
> That's the approach taken in the SSE4 patch, and in fact that's the logical way to do it there. I hadn't considered doing it that way in the pure C case, but I think it's worth trying.

Actually, I spoke too quickly. We can't have an error accumulator in the C case because we need to return how many bytes were valid. In fact, in the SSE case, it checks the error vector at the end and then reruns with the fallback case to count the valid bytes.

--
John Naylor
EDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Andrey Lepikhov
Date:
Subject: Re: Asynchronous Append on postgres_fdw nodes.
Next
From: Pavel Stehule
Date:
Subject: Re: security_definer_search_path GUC