Re: 100% cpu usage on some postmaster processes kill the complete database - Mailing list pgsql-general

From Paul Dunkler
Subject Re: 100% cpu usage on some postmaster processes kill the complete database
Date
Msg-id 18DC6B22-3FEA-4655-94EE-DB3485F6E402@xyrality.com
Whole thread Raw
In response to Re: 100% cpu usage on some postmaster processes kill the complete database  (Richard Huxton <dev@archonet.com>)
Responses Re: 100% cpu usage on some postmaster processes kill the complete database
List pgsql-general
I did that now - and analyzed the situation a bit. There are only queries running which will process very fast under high load (only index scans, very low rates of sequential scans). I found a remarkable number of Insert statements...

And sometimes when that happens, the CPU Utilization is going up to nearby 100% too and 98% is system usage...

Am 01.03.2012 um 18:02 schrieb Richard Huxton:

On 01/03/12 16:41, Paul Dunkler wrote:
Hi List,

we are currently running a rather large postgresql-installation with approximately 4k Transactions and 50k index scans per second.

In the last days on some times of the day (irregular - 3-4 times a day), some of the postmaster processes are running with 100% cpu usage. That leads to a totally breakdown of the query execution. We see tons of statements which are correctly automatically aborted by our statement_timeout set to 15 seconds. I tried to search, but do not really recognize what the problem could be there...

Some things i have checked:
- We are not running any bulk jobs or maintenance scripts at this time
- No system errors in any logs during that slowdowns
- I/O Performance seems fine. No high IO Wait amount... But IO Write totally drops in that times because it seems that no postgres process can perform any update

I just installed a script, which prints me out the top and ps axf information for facing out the problem. I will post a snippet of the top here:

Combine that with this:
 SELECT * FROM pg_stat_activity;

That will let you line up pids from top with active queries.

--
 Richard Huxton
 Archonet Ltd

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Mit freundlichen Grüßen

Paul Dunkler





-----------------------------------------------------
XYRALITY GmbH • Lerchenstraße 28a • 22767 Hamburg
Paul Dunkler • Softwareentwickler
Mail: paul.dunkler@xyrality.com
Tel: +49 (0) 40 23 51 78 97
Mobil: +49 (0) 151 252 228 42
Fax: +49 (0) 40 23 51 78 98
Web: http://www.xyrality.com/
Registergericht: Hamburg HRB 115332
Geschäftsführer: Sven Ossenbrüggen & Alexander Spohr
-----------------------------------------------------

Attachment

pgsql-general by date:

Previous
From: John R Pierce
Date:
Subject: Re: Ident authentication failed for user
Next
From: Richard Huxton
Date:
Subject: Re: 100% cpu usage on some postmaster processes kill the complete database