Re: hung backends stuck in spinlock heavy endless loop - Mailing list pgsql-hackers

From Andres Freund
Subject Re: hung backends stuck in spinlock heavy endless loop
Date
Msg-id 20150115192734.GE14782@awork2.anarazel.de
Whole thread Raw
In response to Re: hung backends stuck in spinlock heavy endless loop  (Andres Freund <andres@2ndquadrant.com>)
List pgsql-hackers
On 2015-01-15 20:15:42 +0100, Andres Freund wrote:
> > WARNING:  did not find subXID 14955 in MyProc
> > CONTEXT:  PL/pgSQL function cdsreconcileruntable(bigint) line 35
> > during exception cleanup
> > WARNING:  you don't own a lock of type RowExclusiveLock
> > CONTEXT:  PL/pgSQL function cdsreconcileruntable(bigint) line 35
> > during exception cleanup
> > LOG:  could not send data to client: Broken pipe
> > CONTEXT:  PL/pgSQL function cdsreconcileruntable(bigint) line 35
> > during exception cleanup
> > STATEMENT:  SELECT CDSReconcileRunTable(2151)
> > WARNING:  ReleaseLockIfHeld: failed??
> > CONTEXT:  PL/pgSQL function cdsreconcileruntable(bigint) line 35
> > during exception cleanup
> > ERROR:  failed to re-find shared proclock object
> > CONTEXT:  PL/pgSQL function cdsreconcileruntable(bigint) line 35
> > during exception cleanup
> > STATEMENT:  SELECT CDSReconcileRunTable(2151)
> > WARNING:  AbortSubTransaction while in ABORT state
> > WARNING:  did not find subXID 14955 in MyProc
> > WARNING:  you don't own a lock of type AccessShareLock
> > WARNING:  ReleaseLockIfHeld: failed??
> > ERROR:  failed to re-find shared proclock object
> > WARNING:  AbortSubTransaction while in ABORT state
> > WARNING:  did not find subXID 14955 in MyProc
> > WARNING:  you don't own a lock of type AccessShareLock
> > WARNING:  ReleaseLockIfHeld: failed??
> > WARNING:  you don't own a lock of type ShareLock
> > TRAP: FailedAssertion("!(FastPathStrongRelationLocks->count[fasthashcode]
> > > 0)", File: "lock.c", Line: 1240)
> > LOG:  server process (PID 10117) was terminated by signal 6: Aborted
> > LOG:  terminating any other active server processes
> 
> Ick.
> 
> Were there any 'LOG: Handling deadlock detected on CdsRunTableId' log
> entries before? It's hard to know from here, but the 'during exception
> cleanup' indicates a problem in abort handling.  Were there any deadlock
> detected errors closeby?

Alternatively were there any 'LOG: CdsRunTableId %s Failed' messages? If
so, what was the cause?

> You're catching deadlock errors in a subtransaction. Hm.

A couple questions:
* Do you also use lock_timeout/statement_timeout? If so, what are their settings + deadlock_timeout?
* were any processes killed at that time?

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: hung backends stuck in spinlock heavy endless loop
Next
From: Bruce Momjian
Date:
Subject: Re: [BUGS] BUG #12070: hstore extension: hstore_to_json_loose produces invalid JSON