Re: group locking: incomplete patch, just for discussion - Mailing list pgsql-hackers
| From | Amit Kapila |
|---|---|
| Subject | Re: group locking: incomplete patch, just for discussion |
| Date | |
| Msg-id | CAA4eK1J-KMn_dww5JdjeZx6dE8EGDXTcg8du0R=F9DGHFRsBCQ@mail.gmail.com Whole thread Raw |
| In response to | Re: group locking: incomplete patch, just for discussion (Robert Haas <robertmhaas@gmail.com>) |
| Responses |
Re: group locking: incomplete patch, just for discussion
(Robert Haas <robertmhaas@gmail.com>)
|
| List | pgsql-hackers |
On Fri, Nov 7, 2014 at 3:55 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Oct 31, 2014 at 9:07 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > Maybe we can, as a first step, make those edges in the lock graph
> > visible to the deadlock detector? It's pretty clear that undetected
> > deadlocks aren't ok, but detectable deadlocks in a couple corner cases
> > might be acceptable.
>
> An interesting point came to mind this morning on this topic, while
> discussing this issue with Amit. Suppose process A1 holds a lock on
> some object and process A2, a member of the same locking group, waits
> for a lock on that same object. It follows that there must exist a
> process B which either *holds* or *awaits* a lock which conflicts with
> the one requested by A2; otherwise, A2 would have been granted the
> lock at once. If B *holds* the conflicting lock, it may eventually
> release it; but if B *awaits* the conflicting lock, we have a soft
> deadlock, because A2 waits for B waits for A1, which we presume to
> wait for A1.[1]
>
> The logical consequence of this is that, if a process seeks to acquire
> a lock which is already held (in any mode) by a fellow group-member,
> it had better jump over the entire wait queue and acquire the lock
> immediately if no conflicting locks have already been granted. If it
> fails to do this, it immediately creates a soft deadlock. What if
> there *are* conflicting locks already granted? In that case, things
> are more complicated. We could, for example, have this situation: A1
> holds AccessShareLock, B holds ExclusiveLock, C1 awaits ShareLock, and
> then (following it in the wait queue) A2 awaits RowExclusiveLock.
> This is not a deadlock, because B can finish, then C can get the lock,
> then C1 can finish, then A2 can get the lock. But if C2 now waits for
> some member of group A, then we have a soft deadlock between group A
> and group C, which can be resolved by moving A2's request ahead of C1.
> (This example is relatively realistic, too: a lock upgrade from
> AccessShareLock to RowExclusiveLock is probably the most common type
> of lock upgrade, for reasons that are hopefully obvious.)
>
> In practice, I suspect it's best to take a more aggressive approach
> and have A2 jump straight to the head of the line right out of the
> chute. Letting some parallel workers make progress while holding up
> others out of a notion of locking fairness is probably stupid, because
> they're probably dependent on each other in some way that makes it
> useless for one to make progress while the others don't. For the same
> reason, I'd argue that we ought to wait until all pending lock
> requests from a given group can be satisfied before granting any of
> them.
>
> On Fri, Oct 31, 2014 at 9:07 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > Maybe we can, as a first step, make those edges in the lock graph
> > visible to the deadlock detector? It's pretty clear that undetected
> > deadlocks aren't ok, but detectable deadlocks in a couple corner cases
> > might be acceptable.
>
> An interesting point came to mind this morning on this topic, while
> discussing this issue with Amit. Suppose process A1 holds a lock on
> some object and process A2, a member of the same locking group, waits
> for a lock on that same object. It follows that there must exist a
> process B which either *holds* or *awaits* a lock which conflicts with
> the one requested by A2; otherwise, A2 would have been granted the
> lock at once. If B *holds* the conflicting lock, it may eventually
> release it; but if B *awaits* the conflicting lock, we have a soft
> deadlock, because A2 waits for B waits for A1, which we presume to
> wait for A1.[1]
>
> The logical consequence of this is that, if a process seeks to acquire
> a lock which is already held (in any mode) by a fellow group-member,
> it had better jump over the entire wait queue and acquire the lock
> immediately if no conflicting locks have already been granted. If it
> fails to do this, it immediately creates a soft deadlock. What if
> there *are* conflicting locks already granted? In that case, things
> are more complicated. We could, for example, have this situation: A1
> holds AccessShareLock, B holds ExclusiveLock, C1 awaits ShareLock, and
> then (following it in the wait queue) A2 awaits RowExclusiveLock.
> This is not a deadlock, because B can finish, then C can get the lock,
> then C1 can finish, then A2 can get the lock. But if C2 now waits for
> some member of group A, then we have a soft deadlock between group A
> and group C, which can be resolved by moving A2's request ahead of C1.
> (This example is relatively realistic, too: a lock upgrade from
> AccessShareLock to RowExclusiveLock is probably the most common type
> of lock upgrade, for reasons that are hopefully obvious.)
>
> In practice, I suspect it's best to take a more aggressive approach
> and have A2 jump straight to the head of the line right out of the
> chute. Letting some parallel workers make progress while holding up
> others out of a notion of locking fairness is probably stupid, because
> they're probably dependent on each other in some way that makes it
> useless for one to make progress while the others don't. For the same
> reason, I'd argue that we ought to wait until all pending lock
> requests from a given group can be satisfied before granting any of
> them.
I think this logic can sometimes block processes from acquiring a lock
which no one is holding. Assume Group-1 (G-1) is waiting on two locks
(Lock L1 on Table T1 and Lock L2 on Table T2) which are held by
unrelated processes P-2 and P-3 respectively; now P-3 releases the
lock on table T-2 and cannot grant it to G-1 who is waiting on this lock
because still L-1 lock cannot be granted to G-1. At this point the situation
is that no one holds lock on T-2 and G-1 waits for it, however any new
process won't be able to acquire lock on T-2 as there is already a waiter
for the same in wait queue. OTOH if we would have granted lock for
T-2 to G-1, it might have finished it's work and released the lock
(considering it is a short time lock on catalog relation).
If it is possible to have such a situation, then even though it is beneficial
for certain cases to view several pending requests from a same group
as single request, at the same time it can be harmful for some unrelated
processes as well.
pgsql-hackers by date: