I wrote:
> Oh, interesting. The largest possible value of "base" is 256, and the
> code limits the amount of string it'll scan to 20 bytes, which means
> "denom" could reach at most 256^21 = 3.7e50. Perfectly fine with
> IEEE-math doubles, but not so much with other arithmetics.
> We could hold the worst-case value to within the VAX range if we
> considered only about 14 bytes instead of 20. Probably that wouldn't
> lose much in terms of estimation accuracy, but given the lack of
> complaints to date, I'm not sure we should change it ...
On further reflection, there seems little reason not to change it: it's
pretty silly to imagine that selectivity estimates produced via this
technique would have anything like 14 decimal places of precision anyhow.
We've already stripped off any common prefix of the strings we're
comparing, so the strings are certain to differ at the first byte.
Reducing the maximum number of bytes considered will save cycles and I
really doubt that it would cost anything in estimation accuracy.
regards, tom lane
