On Fri, Jun 15, 2012 at 6:10 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I would be concerned about this if it were per-sort-tuple wastage, but
> what I understood to be happening was that this was a single instance of
> an expansible buffer (per sort, perhaps, but still just one buffer).
> And, as you keep pointing out when it suits your argument, it's
> relatively uncommon to be sorting enormous values anyway. So no, I am
> not particularly worried about that. If you are, there are more
> important places to be concerned about allocation pad wastage, starting
> with palloc itself.
And, of course, palloc doesn't use power-of-two allocation up to
infinity either, as proposed here; beyond a certain limit it switches
to calling malloc(), which doesn't use power-of-two allocations for
large requests either, at least not in glibc - for anything 128kB or
above, it uses mmap to grab something of exactly the requested size
(rounded up to the OS page size) and returns it the OS unconditionally
when it's freed.
However, what this really boils down to is that you and Peter don't
like this line of code:
+ tss->buflen1 = TYPEALIGN(TEXTBUFLEN, len1);
What would you like it to say instead?
The obvious formulation is:
while (len1 < tss->buflen1) tss->buflen *= 2;
Or perhaps the following, which will normally be more efficient,
though possibly not as efficient as what I've got there now:
tss->buflen = 1 << ffs(len1);
Of course if len1 happens to be large enough that the high-order bit
is set, the first proposal above will fail to terminate and the second
one will produce 0. That can't really happen in practice because a
toasted datum is limited to 1GB, of course, and I suppose a lot of
code would need to be adjusted if we ever wanted to raise that limit,
so maybe it's not worth worrying about it.
So, is one of the above code fragments what people want? Or something else?
Thanks,
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
