Re: Improved ICU patch - WAS: Implementing full UTF-8 support (aka supporting 0x00) - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: Improved ICU patch - WAS: Implementing full UTF-8 support (aka supporting 0x00)
Date
Msg-id CAM3SWZRVBUZ2pdcJ_Br7J+wzZv0m1m4sM=1uFgD+JWh8u8wgXw@mail.gmail.com
Whole thread Raw
In response to Re: Improved ICU patch - WAS: Implementing full UTF-8 support (aka supporting 0x00)  (Palle Girgensohn <girgen@pingpong.net>)
List pgsql-hackers
On Thu, Aug 11, 2016 at 4:22 AM, Palle Girgensohn <girgen@pingpong.net> wrote:
> But in your strxfrm code in PostgreSQL, the keys are cached, and represented
> as int64:s if I remember correctly, so perhaps there is still a benefit
> using the abbreviated keys? More testing is required, I guess...

ICU's ucol_nextSortKeyPart() interface is faster for this, I believe,
and works because you only need the first sizeof(Datum) bytes (4 or 8
bytes). So, you have the right idea about using it (at least for the
abbreviated key stuff), but the second last argument needs to be
sizeof(Datum).

The whole point of the abbreviated key optimization is that you can
avoid pointer chasing during each and every comparison. Your test here
is invalid because it doesn't involved the reuse of the keys. Often,
during a sort, each item has to be compared about 20 times.

I've experimented with ICU, and know it works well with this. You
really need to create a scenario with a real sort, and all the
conditions I describe.

-- 
Peter Geoghegan



pgsql-hackers by date:

Previous
From: Ryan Pedela
Date:
Subject: Re: 9.6 phrase search distance specification
Next
From: Claudio Freire
Date:
Subject: Re: Heap WARM Tuples - Design Draft