Re: [PATCHES] update i386 spinlock for hyperthreading - Mailing list pgsql-hackers

"Simon Riggs" <simon@2ndquadrant.com> writes:
>>Kenneth Marshall would like me to post this:
>> I agree that in order to manage today's large memory machines, we
>> need to have less contention in our buffer management strategies.
>> The two main main choke points are in the buffer hash table routines
>> and in the buffer management linked lists. Unfortunately most of the
>> code depends on holding the bufmgr lock on entry which eliminates
>> many chances for parallelism.

Are you familiar with the work I've been doing recently to try to
reduce the contention for the BufMgrLock? For example:

http://www.mail-archive.com/pgsql-hackers%40postgresql.org/msg40289.html

The approach I've taken is to remove the usage of the BufMgrLock for
operations that do not affect the global state of the buffer pool.
That means that operations like incrementing a buffer's refcount
requires only holding the per-buffer meta data lock. That's only one
part of the puzzle, however: other ways to reduce BufMgrLock
contention will probably be necessary.

Unfortunately this code is not in CVS yet: I've been too busy with
school to wrap up the remaining issues it has. However, I hope to get
it into the tree reasonably soon, and certainly in time for 7.5.

>> The number of buffer pools should at the very minimum be equal to
>> the number of processors in the system. [...]

Not sure I understand exactly what you're suggesting here. Can you
elaborate?

>> The next item to address is the buf_table concurrency. It appears
>> that the same code that was used in the hash index update by Tom
>> Lane could be used to split the buf_table accesses into a
>> per-bucket access using a per-bucket lock and not a global
>> lock. Modifying the current dyn_hash search and update code would
>> make it look effectively like Mr. Lane's new hash index code.

Interesting. This would be complementary, of course, to my work on
changing the buffer locking scheme: perhaps once that is done, we can
reassess the remaining lock contention issues in the bufmgr, and
implement this if necessary?

Another related idea that Jan Wieck and I had discussed was avoiding
acquiring the BufMgrLock at all in BufferAlloc() where possible. For
instance, we could enhance the existing PrivateRefCount mechanism, or
invent some new mechanism, which would essentially keep a LRU list of
buffer tag -> buffer id mappings in each backend's local memory. Then,
we would walk this list in BufferAlloc(): if the buffer tag we're
looking for is already there, we can immediately acquire the buffer's
per-buffer meta data lock (without ever acquiring the BufMgrLock).
We'd need to then check that the buffer hasn't changed under our feet
(compare the locked buffer's tag with what we think its tag should be,
and start over if its different).

-Neil