Re: [PATCHES] update i386 spinlock for hyperthreading - Mailing list pgsql-hackers

On Fri, Feb 20, 2004 at 05:26:46AM -0500, Neil Conway wrote:
> "Simon Riggs" <simon@2ndquadrant.com> writes:
> >>Kenneth Marshall would like me to post this:
> >> I agree that in order to manage today's large memory machines, we
> >> need to have less contention in our buffer management strategies.
> >> The two main main choke points are in the buffer hash table routines
> >> and in the buffer management linked lists. Unfortunately most of the
> >> code depends on holding the bufmgr lock on entry which eliminates
> >> many chances for parallelism.
> 
> Are you familiar with the work I've been doing recently to try to
> reduce the contention for the BufMgrLock? For example:
> 
> http://www.mail-archive.com/pgsql-hackers%40postgresql.org/msg40289.html
> 
> The approach I've taken is to remove the usage of the BufMgrLock for
> operations that do not affect the global state of the buffer pool.
> That means that operations like incrementing a buffer's refcount
> requires only holding the per-buffer meta data lock. That's only one
> part of the puzzle, however: other ways to reduce BufMgrLock
> contention will probably be necessary.
> 
> Unfortunately this code is not in CVS yet: I've been too busy with
> school to wrap up the remaining issues it has. However, I hope to get
> it into the tree reasonably soon, and certainly in time for 7.5.
> 
> >> The number of buffer pools should at the very minimum be equal to
> >> the number of processors in the system. [...]
> 
> Not sure I understand exactly what you're suggesting here. Can you
> elaborate?
> 
> >> The next item to address is the buf_table concurrency. It appears
> >> that the same code that was used in the hash index update by Tom
> >> Lane could be used to split the buf_table accesses into a
> >> per-bucket access using a per-bucket lock and not a global
> >> lock. Modifying the current dyn_hash search and update code would
> >> make it look effectively like Mr. Lane's new hash index code.
> 
> Interesting. This would be complementary, of course, to my work on
> changing the buffer locking scheme: perhaps once that is done, we can
> reassess the remaining lock contention issues in the bufmgr, and
> implement this if necessary?
> 
> Another related idea that Jan Wieck and I had discussed was avoiding
> acquiring the BufMgrLock at all in BufferAlloc() where possible. For
> instance, we could enhance the existing PrivateRefCount mechanism, or
> invent some new mechanism, which would essentially keep a LRU list of
> buffer tag -> buffer id mappings in each backend's local memory. Then,
> we would walk this list in BufferAlloc(): if the buffer tag we're
> looking for is already there, we can immediately acquire the buffer's
> per-buffer meta data lock (without ever acquiring the BufMgrLock).
> We'd need to then check that the buffer hasn't changed under our feet
> (compare the locked buffer's tag with what we think its tag should be,
> and start over if its different).
> 
> -Neil
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend

Neil,

I have been following the discussion in the pgsql-hackers list. I tried
to apply the patch you mentioned above, but I did not have the same
version of postgres and had a lot of rejects. I also wanted to see your
approach to adding a finer-grained lock structure to the buffer manager;
since some of my ideas would depend on the implimentation used.

My comment on the number of buffer pools:

>> The number of buffer pools should at the very minimum be equal to
>> the number of processors in the system. [...]

refers to the fact that if you could provide a per-CPU buffer pool
you would be able to minimize the intra-CPU cache sync. The code
would need to be able to find out what CPU it was running on to make
that work. Other wise, simply splitting the buffer pool into several
pools with a per-pool lock would increase the concurrency proportional
to the number of pools. The buffer header would have a pool id to
allow you to grab the appropriate per-pool lock. Also preferentially
take a new buffer from the pool you are already using.

I am waiting for your commit to CVS to look at if further. If you
think that will be a while still, could you let me know which version
of postgres I can use to get a clean patch installation from 

http://www.mail-archive.com/pgsql-hackers%40postgresql.org/msg40289.html

--Ken