Re: Rather large LA - Mailing list pgsql-performance

From Richard Shaw
Subject Re: Rather large LA
Date
Msg-id 9BFA2771-CA9B-4EEF-B671-78FF67F749C6@aggress.net
Whole thread Raw
In response to Re: Rather large LA  (Alan Hodgson <ahodgson@simkin.ca>)
Responses Re: Rather large LA  (Scott Marlowe <scott.marlowe@gmail.com>)
Re: Rather large LA  (Alan Hodgson <ahodgson@simkin.ca>)
Re: Rather large LA  (Vitalii Tymchyshyn <tivv00@gmail.com>)
List pgsql-performance
vmstat 1 and iostat -x output

Normal


procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 3  0   2332 442428  73904 31287344    0    0    89    42    0    0  7  5 85  3  0
 4  1   2332 428428  73904 31288288    0    0  1440     0 6553 29066  5  2 91  1  0
 4  1   2332 422688  73904 31288688    0    0   856     0 4480 18860  3  1 95  1  0
 0  0   2332 476072  73920 31289444    0    0   544  1452 4478 19103  3  1 95  0  0
 3  0   2332 422288  73920 31290572    0    0  1268   496 5565 23410  5  3 91  1  0

cavg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.11    0.01    2.58    2.56    0.00   89.74

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               1.00   143.00 523.50 108.00  8364.00  2008.00    16.42     2.78    4.41   1.56  98.35
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              1.00   143.00 523.50 108.00  8364.00  2008.00    16.42     2.78    4.41   1.56  98.35
sda3              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           4.89    0.00    2.94    3.14    0.00   89.04

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               1.00     0.00 285.00  0.00  4808.00     0.00    16.87     2.46    8.29   3.02  86.20
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              1.00     0.00 285.00  0.00  4808.00     0.00    16.87     2.46    8.29   3.02  86.20
sda3              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb               0.00   161.50  0.00  6.50     0.00  1344.00   206.77     0.00    0.69   0.15   0.10
sdb1              0.00   161.50  0.00  6.50     0.00  1344.00   206.77     0.00    0.69   0.15   0.10


After Restart

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2 34   2332 5819012  75632 25855368    0    0    89    42    0    0  7  5 85  3  0
 4 39   2332 5813344  75628 25833588    0    0  5104   324 5480 27047  3  1 84 11  0
 2 47   2332 5815212  75336 25812064    0    0  4356  1664 5627 28695  3  1 84 12  0
 2 40   2332 5852452  75340 25817496    0    0  5632   828 5817 28832  3  1 84 11  0
 1 45   2332 5835704  75348 25817072    0    0  4960  1004 5111 25782  2  1 88  9  0
 2 42   2332 5840320  75356 25811632    0    0  3884   492 5405 27858  3  1 88  8  0
 0 47   2332 5826648  75348 25805296    0    0  4432  1268 5888 29556  3  1 83 13  0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.26    0.00    1.69   25.21    0.00   69.84

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.50    45.00 520.00  2.50  8316.00   380.00    16.64    71.70  118.28   1.92 100.10
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              0.50    45.00 520.00  2.50  8316.00   380.00    16.64    71.70  118.28   1.92 100.10
sda3              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb               0.00   196.50  0.00 10.50     0.00  1656.00   157.71     0.01    0.67   0.52   0.55
sdb1              0.00   196.50  0.00 10.50     0.00  1656.00   157.71     0.01    0.67   0.52   0.55

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.97    0.00    1.71   20.88    0.00   73.44

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               1.00     0.00 532.00  0.00  8568.00     0.00    16.11    73.73  148.44   1.88 100.05
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              1.00     0.00 532.00  0.00  8568.00     0.00    16.11    73.73  148.44   1.88 100.05
sda3              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb               0.00   106.50  0.00 11.50     0.00   944.00    82.09     0.00    0.39   0.30   0.35
sdb1              0.00   106.50  0.00 11.50     0.00   944.00    82.09     0.00    0.39   0.30   0.35

Regards

Richard

.........

On 5 Sep 2011, at 21:05, Alan Hodgson wrote:

> On September 5, 2011, Richard Shaw <richard@aggress.net> wrote:
> > Hi Andy,
> >
> > It's not a new issue no, It's a legacy system that is in no way ideal but
> > is also not in a position to be overhauled.  Indexes are correct, tables
> > are up to 25 million rows.
> >
> > On startup, it hits CPU more than IO, I'll provide some additional stats
> > after I restart it tonight.
>
> I bet it's I/O bound until a good chunk of the active data gets cached. Run a vmstat 1 while it's that busy, I bet
mostof the cpu time is really in io_wait.  


pgsql-performance by date:

Previous
From: "Tomas Vondra"
Date:
Subject: Re: Sudden drop in DBb performance
Next
From: Scott Marlowe
Date:
Subject: Re: Rather large LA