Re: PATCH: two slab-like memory allocators - Mailing list pgsql-hackers
| From | John Gorman |
|---|---|
| Subject | Re: PATCH: two slab-like memory allocators |
| Date | |
| Msg-id | CALkS6B8hisTEJJ7ZCQ_GiBLNVzKfsevAD+92rYN6myyFK=UQtw@mail.gmail.com Whole thread Raw |
| In response to | Re: PATCH: two slab-like memory allocators (Tomas Vondra <tomas.vondra@2ndquadrant.com>) |
| Responses |
Re: PATCH: two slab-like memory allocators
(Tomas Vondra <tomas.vondra@2ndquadrant.com>)
|
| List | pgsql-hackers |
SlabContext has a parent context. It can delegate requests that
it cannot handle to the parent context which is GenSlab. Genslab
can send them to the sister aset context.
This could handle all reallocs so there will be no surprises.
Remind me again why we cannot realloc in place for sizes
smaller than chunkSize? GenSlab is happy to allocate smaller
sizes and put them into the fixed size chunks.
Maybe SlabAlloc can be happy with sizes up to chunkSize.
if (size <= set->chunkSize)
return MemoryContextAlloc(set->slab, size);
else
return MemoryContextAlloc(set->aset, size);
On Sat, Oct 1, 2016 at 10:15 PM, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
On 10/02/2016 12:23 AM, John Gorman wrote:I reproduced the quadradic pfree performance problem and verified that
these patches solved it.
The slab.c data structures and functions contain no quadradic components.
I noticed the sizing loop in SlabContextCreate() and came up with
a similar formula to determine chunksPerBlock that you arrived at.Firstly, I've realized there's an issue when chunkSize gets too
large - once it exceeds blockSize, the SlabContextCreate() fails
as it's impossible to place a single chunk into the block. In
reorderbuffer, this may happen when the tuples (allocated in
tup_context) get larger than 8MB, as the context uses
SLAB_LARGE_BLOCK_SIZE (which is 8MB).
But maybe there's a simpler solution - we may simply cap the
chunkSize (in GenSlab) to ALLOC_CHUNK_LIMIT. That's fine, because
AllocSet handles those requests in a special way - for example
instead of tracking them in freelist, those chunks got freed
immediately.
I like this approach because it fixes the performance problems
with smaller allocations and doesn't change how larger
allocations are handled.
One more comment about GenSlab, particularly about unpredictability of the repalloc() behavior. It works for "large" chunks allocated in the AllocSet part, and mostly does not work for "small" chunks allocated in the SlabContext. Moreover, the chunkSize changes over time, so for two chunks of the same size, repalloc() may work on one of them and fail on the other, depending on time of allocation.
With SlabContext it's perfectly predictable - repalloc() call fails unless the chunk size is exactly the same as before (which is perhaps a bit pointless, but if we decide to fail even in this case it'll work 100% time).
But with GenSlabContext it's unclear whether the call fails or not.
I don't like this unpredictability - I'd much rather have consistent failures (making sure people don't do repalloc() on with GenSlab). But I don't see a nice way to achieve that - the repalloc() call does not go through GenSlabRealloc() at all, but directly to SlabRealloc() or AllocSetRealloc().
The best solution I can think of is adding an alternate version of AllocSetMethods, pointing to a different AllocSetReset implementation.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
pgsql-hackers by date: