RESOLVED: Re: Wierd context-switching issue on Xeon - Mailing list pgsql-performance

Tom, Josh,

I think we have the problem resolved after I found the following note
from Tom:

 > A large number of semops may mean that you have excessive contention
on some lockable
 > resource, but I don't have enough info to guess what resource.

This was the key to look at: we were missing all indices on table which
is used heavily and does lots of locking. After recreating the missing
indices the production system performed normal. No, more excessive
semop() calls, load way below 1.0, CS over 20.000 very rare, more in
thousands realm and less.

This is quite a relief but I am sorry that the problem was so stupid and
you wasted some time although Tom said he had also seem excessive
semop() calls on another Dual XEON system.

Hyperthreading was turned off so far but will be turned on again the
next days. I don't expect any problems then.

I'm not sure if this semop() problem is still an issue but the database
behaves a bit out of bounds in this situation, i.e. consuming system
resources with semop() calls 95% while tables are locked very often and
longer.

Thanks for your help,

Dirk

At last here is the current vmstat 1 excerpt where the problem has been
resolved:



procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy
id wa
 1  0   2308 232508 201924 6976532    0    0   136   464  628   812  5
1 94  0
 0  0   2308 232500 201928 6976628    0    0    96   296  495   484  4
0 95  0
 0  1   2308 232492 201928 6976628    0    0     0   176  347   278  1
0 99  0
 0  0   2308 233484 201928 6976596    0    0    40   580  443   351  8
2 90  0
 1  0   2308 233484 201928 6976696    0    0    76   692  792   651  9
2 88  0
 0  0   2308 233484 201928 6976696    0    0     0    20  132    34  0
0 100  0
 0  0   2308 233484 201928 6976696    0    0     0    76  177    90  0
0 100  0
 0  1   2308 233484 201928 6976696    0    0     0   216  321   250  4
0 96  0
 0  0   2308 233484 201928 6976696    0    0     0   116  417   240  8
0 92  0
 0  0   2308 233484 201928 6976784    0    0    48   600  403   270  8
0 92  0
 0  0   2308 233464 201928 6976860    0    0    76   452 1064  2611 14
1 84  0
 0  0   2308 233460 201932 6976900    0    0    32   256  587   587 12
1 87  0
 0  0   2308 233460 201932 6976932    0    0    32   188  379   287  5
0 94  0
 0  0   2308 233460 201932 6976932    0    0     0     0  103     8  0
0 100  0
 0  0   2308 233460 201932 6976932    0    0     0     0  102    14  0
0 100  0
 0  1   2308 233444 201948 6976932    0    0     0   348  300   180  1
0 99  0
 1  0   2308 233424 201948 6976948    0    0    16   380  739   906  4
2 93  0
 0  0   2308 233424 201948 6977032    0    0    68   260  724   987  7
0 92  0
 0  0   2308 231924 201948 6977128    0    0    96   344 1130   753 11
1 88  0
 1  0   2308 231924 201948 6977248    0    0   112   324  687   628  3
0 97  0
 0  0   2308 231924 201948 6977248    0    0     0   192  575   430  5
0 95  0
 1  0   2308 231924 201948 6977248    0    0     0   264  208   124  0
0 100  0
 0  0   2308 231924 201948 6977264    0    0    16   272  380   230  3
2 95  0
 0  0   2308 231924 201948 6977264    0    0     0     0  104     8  0
0 100  0
 0  0   2308 231924 201948 6977264    0    0     0    48  258    92  1
0 99  0
 0  0   2308 231816 201948 6977484    0    0   212   268  456   384  2
0 98  0
 0  0   2308 231816 201948 6977484    0    0     0    88  453   770  0
0 99  0
 0  0   2308 231452 201948 6977680    0    0   196   476  615   676  5
0 94  0
 0  0   2308 231452 201948 6977680    0    0     0   228  431   400  2
0 98  0
 0  0   2308 231452 201948 6977680    0    0     0     0  237    58  3
0 97  0
 0  0   2308 231448 201952 6977680    0    0     0     0  365    84  2
0 97  0
 0  0   2308 231448 201952 6977680    0    0     0    40  246   108  1
0 99  0
 0  0   2308 231448 201952 6977776    0    0    96   352  606  1026  4
2 94  0
 0  0   2308 231448 201952 6977776    0    0     0   240  295   266  5
0 95  0