Re: RAID 1 - drive failed - very slow queries even after drive replaced - Mailing list pgsql-general

From Merlin Moncure
Subject Re: RAID 1 - drive failed - very slow queries even after drive replaced
Date
Msg-id AANLkTimQ=DYcBjfeRVn2QOVT4-xghzaML3mt1bDLDCCt@mail.gmail.com
Whole thread Raw
In response to RAID 1 - drive failed - very slow queries even after drive replaced  (Merrick <merrick@gmail.com>)
List pgsql-general
On Wed, Mar 23, 2011 at 3:33 AM, Merrick <merrick@gmail.com> wrote:
> Hi,
>
> I am looking for some advice on where to troubleshoot after 1 drive in
> a RAID 1 failed.
>
> Thank you.
>
> I am running v 7.41, I am currently importing the data to another
> physical server running 8.4 and will test with that once I can. In the
> meantime here is relevant info:
>
> Backups used to take 25 minutes, and now take 110 minutes, before
> replacing the drive it became clear the backup was not going to finish
> since in 120 minutes it had only finished 200mb of 2.8gb.
>
> Before replacing the drive:
> -----------------------------------
> We noticed all of the queries were slow, many taking over 100 seconds.
> After we replaced the drives we noticed the queries are running 40
> seconds or more and most are 8 seconds or more where the same query
> used to take only 1 second. We have replaced a drive in this RAID 1
> before and nothing like this happened. The schema was not touched for
> at least 1 week prior to this.
>
> Since replacing the drive I have:
> -------------------------------------------
> Restored from a backup a few hours before the queries became very
> slow.
> Reindex all tables
> Vacuum all tables
> Analyze all tables
>
> Here is what I get with iostat:
>
> iostat -k /dev/sda2
> Linux 2.6.26-2-686-bigmem (db1)
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>          19.61    0.00    8.34    1.60    0.00   70.45

probably the replacement drive is bunk, or some esoteric hw problem is
tripping you up.  some iostat numbers while you are having the problem
would be more telling.  the solution is obvious -- in terms of this
server, it's time to ramble on...

merlin

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: Weird problems with C extension and bytea as input type
Next
From: Adrian Klaver
Date:
Subject: Re: pg_dump problems