Re: Seeking performance advice and explanation for high I/O on 8.3 - Mailing list pgsql-performance

From Andy Colson
Subject Re: Seeking performance advice and explanation for high I/O on 8.3
Date
Msg-id 4AA03074.1000107@squeakycode.net
Whole thread Raw
In response to Re: Seeking performance advice and explanation for high I/O on 8.3  ("Scott Otis" <scott.otis@intand.com>)
List pgsql-performance
Scott Otis wrote:
> Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list.
>
> Would providing more information about the size and complexities of the databases help?
>
> I measure I/O stats with iostat - here is the command I use:
>
> iostat -d -x mfid0 -t 290 2
>
> I tried looking at the man page for iostat but couldn't find anywhere how to determine what the stats are for
sequentialvs random - any help there? 
>
> When using 'top -m io' the postgres stats collector process is constantly at 99% - 100%.
>
> When using 'top' the WCPU for the postgres stats collector and the autovacuum process are constantly at 20% - 21%.
>
> Is that normal?  It seems to me that the stats collector is doing all the I/O (which would mean the stats collector
isdoing 46.1 megabytes /sec). 
>
> Also, the I/O stats don't change hardly at all (except at night during backups which makes sense).  They don't go up
ordown with user activity on the server - which makes me wonder a little bit.  I have a feeling that if I just turned
offApache that the I/O stats wouldn't change.  Which leads me to believe that the I/O is not query related - its stats
collectingand autovacuuming related.  Is that expected? 
>
> It seems to me that the stats collector shouldn't be using that much I/O and CPU (and the autovacuum shouldn't be
usingthat much CPU)  - therefore something in my configuration must be messed up or could be changed somehow.  But
maybeI'm wrong - please let me know. 
>
> I don't think my setup is necessarily slow.  I just want to make it as efficient as possible and wanted to get some
feedbackto see if am setting things up right.  I am also looking out into the future and seeing how much load I can put
onthis server before getting another one.  If I can reduce the I/O and CPU that the stats collector and autovacuum are
usingwithout losing any functionality then I can put more load on the server. 
>
> Again thanks for all the help.
>
> Scott Otis
> CIO / Lead Developer
> Intand
> www.intand.com
>

> When using 'top -m io' the postgres stats collector process is constantly at 99% - 100%.
> When using 'top' the WCPU for the postgres stats collector and the autovacuum process are constantly at 20% - 21%.

Yeah, that sounds excessive.  But my database gets 20 transactions a DAY, so, I have no experience with a busy box.

You say its mostly selects, but do you have any triggers or anything that might update a table?  Do you do inserts or
updatesto track traffic? 

What does:

select * from pg_stat_activity

look like? (I think vacuum will show up in there, right?)  I'm curious if we can find the table autovacuum is working
on,maybe that'll help pin it down. 

-Andy

pgsql-performance by date:

Previous
From: "Scott Otis"
Date:
Subject: Re: Seeking performance advice and explanation for high I/O on 8.3
Next
From: Mark Starkman
Date:
Subject: PostgreSQL not using index for statement with group by