Thread: [WIP] shared locks
Hackers, Here is another attempt at the shared locks patch. This is a radically different approach, using Xmax in the tuples and SLRU areas instead of the lock manager. The idea is that a tuple's Xmax can either be a real TransactionId (which is used normally like current CVS tip), or, if the infomask has HEAP_XMAX_SHARED_LOCK, a MultiXactId. A MultiXactId is an abstraction for a set of TransactionIds. So a locker can sleep on the set (effectively calling XactLockTableWait on each member), add itself to a previously existing set, or create a new set with only itself. MultiXactIds are implemented using two SLRU areas and a couple of variables in ShmemVariableCache. We also XLog groups of them just like we do for Oids. This patch is a work in progress. I need to test it more, but the basic infrastructure is written and working. I'd love some comments on the basic design idea. The patch applies cleanly to current CVS tip. There are two new files which are src/backend/access/transam/multixact.c and src/include/access/multixact.h, included separately. Thanks, -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) "I dream about dreams about dreams", sang the nightingale under the pale moon (Sandman)
Attachment
Alvaro Herrera <alvherre@dcc.uchile.cl> writes: > The idea is that a tuple's Xmax can either be a real TransactionId > (which is used normally like current CVS tip), or, if the infomask has > HEAP_XMAX_SHARED_LOCK, a MultiXactId. Interesting idea. Would it be possible to invoke this mechanism only when actually needed --- that is, the first locker of a given tuple puts his plain TransactionId into Xmax (and also sets an infomask bit indicating his intent to have a shared rather than exclusive lock), and then the second locker to come along replaces the TransactionId with a MultiTransactionId including himself and the first locker? This requires 2 infomask bits: 1 for shared vs exclusive lock and one for whether the Xmax contains a TransactionId or MultiTransactionId. But we have them available, and I think I like keeping those concepts separate anyway. (Who's to say we wouldn't want to allow a MultiTransactionId to hold an exclusive lock, someday?) The advantage of course would be substantially less overhead in the very common case where there's no actual contention for the tuple. > MultiXactIds are implemented using two SLRU areas and a couple of > variables in ShmemVariableCache. We also XLog groups of them just like > we do for Oids. So no need for expansible shmem storage? Might be the way to go. I haven't read the patch yet but the idea sounds promising. regards, tom lane
Tom Lane wrote: > Alvaro Herrera <alvherre@dcc.uchile.cl> writes: > > The idea is that a tuple's Xmax can either be a real TransactionId > > (which is used normally like current CVS tip), or, if the infomask has > > HEAP_XMAX_SHARED_LOCK, a MultiXactId. > > Interesting idea. Would it be possible to invoke this mechanism only > when actually needed --- that is, the first locker of a given tuple > puts his plain TransactionId into Xmax (and also sets an infomask bit > indicating his intent to have a shared rather than exclusive lock), > and then the second locker to come along replaces the TransactionId > with a MultiTransactionId including himself and the first locker? > > This requires 2 infomask bits: 1 for shared vs exclusive lock and one > for whether the Xmax contains a TransactionId or MultiTransactionId. > But we have them available, and I think I like keeping those concepts > separate anyway. (Who's to say we wouldn't want to allow a > MultiTransactionId to hold an exclusive lock, someday?) > > The advantage of course would be substantially less overhead in the very > common case where there's no actual contention for the tuple. Yes, that is certainly possible. Alvaro felt he wanted something simpler and that the two-bit case would add complexity, but I agree it would reduce overhead in the most common case. > > MultiXactIds are implemented using two SLRU areas and a couple of > > variables in ShmemVariableCache. We also XLog groups of them just like > > we do for Oids. > > So no need for expansible shmem storage? Might be the way to go. > I haven't read the patch yet but the idea sounds promising. Right. What he does is to use something like pg_subtrans to have a rolling window of current multi-xid sets and the idea is that most access will fall into a small window that is easily stored in the memory window. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
On Mon, Apr 18, 2005 at 09:53:38PM -0400, Bruce Momjian wrote: > Tom Lane wrote: > > Alvaro Herrera <alvherre@dcc.uchile.cl> writes: > > > The idea is that a tuple's Xmax can either be a real TransactionId > > > (which is used normally like current CVS tip), or, if the infomask has > > > HEAP_XMAX_SHARED_LOCK, a MultiXactId. > > > > Interesting idea. Would it be possible to invoke this mechanism only > > when actually needed --- that is, the first locker of a given tuple > > puts his plain TransactionId into Xmax (and also sets an infomask bit > > indicating his intent to have a shared rather than exclusive lock), > > and then the second locker to come along replaces the TransactionId > > with a MultiTransactionId including himself and the first locker? > > > > This requires 2 infomask bits: 1 for shared vs exclusive lock and one > > for whether the Xmax contains a TransactionId or MultiTransactionId. > > But we have them available, and I think I like keeping those concepts > > separate anyway. (Who's to say we wouldn't want to allow a > > MultiTransactionId to hold an exclusive lock, someday?) > > > > The advantage of course would be substantially less overhead in the very > > common case where there's no actual contention for the tuple. > > Yes, that is certainly possible. Alvaro felt he wanted something > simpler and that the two-bit case would add complexity, but I agree it > would reduce overhead in the most common case. I had thought it would make things more complicated. Now that I know how the whole thing works I can handle the extra complexity, which is not much really. Also I wasn't sure if we wanted to waste two infomask bits on this :-) > > > MultiXactIds are implemented using two SLRU areas and a couple of > > > variables in ShmemVariableCache. We also XLog groups of them just like > > > we do for Oids. > > > > So no need for expansible shmem storage? Might be the way to go. Right. I have stashed some info (like next MultiXactId to assign, the first MultiXactId this transaction was assigned, etc) in ShmemVariableCache and PGPROC, but I'm now thinking in storing it in a [fixed size] shmem area private to multixact.c; this way I don't have to lock SInvalLock. BTW, I had to use three additional LWLocks: two for SLRU and one for MultiXactId generation, which also covers the ShmemVariableCache variables. I hope that's OK. -- Alvaro Herrera (<alvherre[@]dcc.uchile.cl>) "No es bueno caminar con un hombre muerto"
On Mon, Apr 18, 2005 at 08:00:57PM -0400, Tom Lane wrote: > Alvaro Herrera <alvherre@dcc.uchile.cl> writes: > > The idea is that a tuple's Xmax can either be a real TransactionId > > (which is used normally like current CVS tip), or, if the infomask has > > HEAP_XMAX_SHARED_LOCK, a MultiXactId. > > Interesting idea. Would it be possible to invoke this mechanism only > when actually needed --- that is, the first locker of a given tuple > puts his plain TransactionId into Xmax (and also sets an infomask bit > indicating his intent to have a shared rather than exclusive lock), > and then the second locker to come along replaces the TransactionId > with a MultiTransactionId including himself and the first locker? Ok, here is the patch again. I did this, so there are now two related bits in the infomask: HEAP_XMAX_IS_MULTI and HEAP_XMAX_{SHARED,EXCLUSIVE}_LOCK. (I ripped out HEAP_XMAX_FOR_UPDATE). Locking and using a MultiXactId are orthogonal. The rest is more or less the same that was in the original patch. I feel this is in a OK state for review for possible inclusion. Some testing is still needed regarding MultiXactId wraparound, and SLRU truncation, and I haven't looked at whether documentation needs updating. -- Alvaro Herrera (<alvherre[@]dcc.uchile.cl>) Jude: I wish humans laid eggs Ringlord: Why would you want humans to lay eggs? Jude: So I can eat them