Tomas Vondra <tv@fuzzy.cz> writes:
> On 2.12.2014 01:33, Tom Lane wrote:
>> What I suspect you're looking at here is the detritus of creation of
>> a huge number of memory contexts. mcxt.c keeps its own state about
>> existing contents in TopMemoryContext. So, if we posit that those
>> contexts weren't real small, there's certainly room to believe that
>> there was some major memory bloat going on recently.
> Aha! MemoryContextCreate allocates the memory for the new context from
> TopMemoryContext explicitly, so that it survives resets of the parent
> context. Is that what you had in mind by keeping state about existing
> contexts?
Right.
> That'd probably explain the TopMemoryContext size, because array_agg()
> creates separate context for each group. So if you have 1M groups, you
> have 1M contexts. Although I don's see how the size of those contexts
> would matter?
Well, if they're each 6K, that's your 6GB right there.
> Maybe we could move this info (list of child contexts) to the parent
> context somehow, so that it's freed when the context is destroyed?
We intentionally didn't do that, because in many cases it'd result in
parent contexts becoming nonempty when otherwise they'd never have
anything actually in them. The idea was that such "shell" parent contexts
should be cheap, requiring only a control block in TopMemoryContext and
not an actual allocation arena. This idea of a million separate child
contexts was never part of the design of course; we might need to rethink
whether that's a good idea. Or maybe there need to be two different
policies about where to put child control blocks.
> Also, this explains the TopMemoryContext size, but not the RSS size (or
> am I missing something)?
Very possibly you're left with "islands" that prevent reclaiming very
much of the peak RAM usage. It'd be hard to be sure without some sort
of memory map, of course.
regards, tom lane
