Re: spinlocks storm bug - Mailing list pgsql-hackers

From Andres Freund
Subject Re: spinlocks storm bug
Date
Msg-id 20131206095629.GI7814@awork2.anarazel.de
Whole thread Raw
In response to spinlocks storm bug  (Pavel Stehule <pavel.stehule@gmail.com>)
Responses Re: spinlocks storm bug
List pgsql-hackers
On 2013-12-06 07:22:27 +0100, Pavel Stehule wrote:
> I have a report of critical bug (database is temporary unavailability ..
> restart is necessary).

> PostgreSQL 9.2.4,
> 24 CPU
> 140G RAM
> SSD disc for all
> 
> 
> Database is under high load. There is a few databases with very high number
> of similar simple statements. When application produce higher load, then
> number of active connection is increased to 300-600 about.
> 
> In some moment starts described event - there is a minimal IO, all CPU are
> on 100%.
> 
> Perf result shows:
>            354246.00 93.0% s_lock
> /usr/lib/postgresql/9.2/bin/postgres
>             10503.00  2.8% LWLockRelease
>  /usr/lib/postgresql/9.2/bin/postgres
>              8802.00  2.3% LWLockAcquire

> We try to limit a connection to 300, but I am not sure if this issue is not
> related to some Postgres bug.

We've seen this issue repeatedly now. None of the times it turned out to
be a bug, but just limitations in postgres' scalability. If you can I'd
strongly suggest trying to get postgres binaries compiled with
-fno-omit-frame-pointer installed to check which locks are actually
conteded.
My bet is BufMappingLock.

There's a CF entry about changing our lwlock implementation to scale
better...

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Boszormenyi Zoltan
Date:
Subject: Re: Backup throttling
Next
From: "Etsuro Fujita"
Date:
Subject: Re: Show lossy heap block info in EXPLAIN ANALYZE for bitmap heap scan