Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding
Date
Msg-id CAEepm=1YRLKbTuhDOEHyRfDu+0qravTuZU0s9+UDB_EZD03KxQ@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Fri, Nov 24, 2017 at 4:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> It doesn't seem that crazy to expose aset.c's overhead size to client
>> code does it?  Most client code wouldn't care, but things that are
>> doing something closer to memory allocator work themselves like
>> dense_alloc could care.  It could deal with its own overhead itself,
>> and subtract aset.c's overhead using a macro.
>
> Seeing that we now have several allocators with different overheads,
> I think that exposing this directly to clients is exactly what not to do.
> I still like the idea I sketched above of a context-type-specific function
> to adjust a request size to something that's efficient.
>
> But there's still the question of how do we know what's an efficient-sized
> malloc request.  Is there good reason to suppose that powers of 2 are OK?

It looks like current glibc versions don't do that sort of thing until
you reach M_MMAP_THRESHOLD (default 128kB) and then starts working in
4kb pages.  Before that its manual says that it doesn't do any
rounding beyond alignment[1].  I haven't come across any other
allocator that is so forgiving, but I haven't checked Illumos, AIX or
Windows.  Only the first of those has source code available, but they
confuse you by shipping a bunch of different allocators in their tree.
I didn't check the other BSDs.  tcmalloc[2], jemalloc, macOS malloc
round up to (respectively) 4k page, 8kb size class, 512 byte size
class when you allocate 32kb + 1 byte.

[1] https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html
[2] http://goog-perftools.sourceforge.net/doc/tcmalloc.html

-- 
Thomas Munro
http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: atorikoshi
Date:
Subject: Re: Failed to delete old ReorderBuffer spilled files
Next
From: Ashutosh Bapat
Date:
Subject: Re: [HACKERS] Partition-wise aggregation/grouping