We've had a TODO item for some time about allowing the user to set the
size of the local buffer array that's used for accessing temporary
tables. The holdup has been that localbuf.c used very unscalable
algorithms (like linear search) and so a large local buffer set would
have terrible performance anyway. We wanted localbuf.c to duplicate the
shared buffer manager's search and replacement algorithms, which looked
like a lot of work.
However, the recent changes to make the shared buffer manager use a
clock sweep replacement algorithm made it trivial to have localbuf.c
do the same. I have just committed additional changes to make
localbuf.c use a hash table instead of linear search for lookup,
so it's now fully on par with the shared buffer manager as far
as algorithms go.
That means we can go ahead with providing a GUC variable to make the
array size user-selectable. I was thinking of calling it either
"local_buffers" (in contrast to "shared_buffers") or "temp_buffers"
(to emphasize the fact that they're used for temporary tables).
Anyone have a preference, or a better alternative?
As far as semantics go, I was thinking of making the variable USERSET
but allowing it to change only as long as you haven't accessed any temp
tables in the current session. Under the hood, we'd postpone calling
InitLocalBuffer() until the first use of temp tables in a session,
at which time the local buffer descriptor array would be allocated,
and henceforth you couldn't change the array size anymore. This would
be enough flexibility to allow temp-table-intensive tasks to run with
a large local setting, without having to make every session do the same.
(It's conceivable that we could support on-the-fly resizing of the
array, but it seems unlikely to be worth the trouble and risk of bugs.)
It's already true that the individual buffers, as opposed to the buffer
descriptors, are allocated only as needed; which makes the overhead
of a large local_buffers setting pretty small if you don't actually do
much with temp tables in a given session. So I was thinking about
making the default value fairly robust, maybe 1000 (as compared to
the historical value of 64...).
Comments?
regards, tom lane
