Another misbehavior that I noted while playing with Artur's example is
that, while GIN index build seems to adhere pretty well to whatever
maintenance_work_mem limit you give it in 8.4, it blows right by that
limit in 9.0 and HEAD --- I observed it happily eating something north
of 128MB with a maintenance_work_mem of 70MB. It looks to me like the
reason is the new "rbtree.c" code, which adds a whole lot of data
structure that's not being counted against the maintenance_work_mem
limit.
Now the first question that might be asked is what we'd need to do to
rbtree.c's API to allow its memory consumption to be tracked. However,
I think there's a considerably more pressing question, which is what
is it that rbtree.c is doing that justifies a 2X bloat factor in GIN
index build --- which is pretty much what it's costing us, given the
observation above. We know that the amount of memory available for
the build has an enormous effect on build time. If we just do the
obvious thing of including the rbtree data structure in the
maintenance_work_mem calculation, what we're going to get is a very
substantial slowdown in build speed for the same maintenance_work_mem
setting, compared to the way 8.4 worked.
So, I would like somebody to show cause why that whole module shouldn't
be ripped out and the code reverted to where it was in 8.4. My
recollection is that the argument for adding it was to speed things up
in corner cases, but what I think it's actually going to do is slow
things down in every case.
regards, tom lane
