Re: why does swap not recover? - Mailing list pgsql-performance

From Scott Carey
Subject Re: why does swap not recover?
Date
Msg-id F7A0C72B-20D5-4A81-853F-014ACC9BA416@richrelevance.com
Whole thread Raw
In response to why does swap not recover?  (Richard Yen <dba@richyen.com>)
Responses Re: why does swap not recover?  (Richard Yen <dba@richyen.com>)
List pgsql-performance
On Mar 26, 2010, at 4:57 PM, Richard Yen wrote:

> Hi everyone,
>
> We've recently encountered some swapping issues on our CentOS 64GB Nehalem machine, running postgres 8.4.2.
Unfortunately,I was foolish enough to set shared_buffers to 40GB.  I was wondering if anyone would have any insight
intowhy the swapping suddenly starts, but never recovers? 
>
> <img src="http://richyen.com/i/swap.png">
>
> Note, the machine has been up and running since mid-December 2009.  It was only a March 8 that this swapping began,
andit's never recovered. 
>
> If we look at dstat, we find the following:
>
> <img src="http://richyen.com/i/dstat.png">
>
> Note that it is constantly paging in, but never paging out.  This would indicate that it's constantly reading from
swap,but never writing out to it.  Why would postgres do this? (postgres is pretty much the only thing running on this
machine).
>
> I'm planning on lowering the shared_buffers to a more sane value, like 25GB (pgtune recommends this for a
Mixed-purposemachine) or less (pgtune recommends 14GB for an OLTP machine).  However, before I do this (and possibly
resolvethe issue), I was hoping to see if anyone would have an explanation for the constant reading from swap, but
neverwriting back. 

Linux until recently does not account for shared memory properly in its swap 'aggressiveness' decisions.
Setting shared_buffers larger than 35% is asking for trouble.

You could try adjusting the 'swappiness' setting on the fly and seeing how it reacts, but one consequence of that is
tradingoff disk swapping for kswapd using up tons of CPU causing other trouble. 

Either use one of the last few kernel versions (I forget which addressed the memory accounting issues, and haven't
triedit myself), or turn shared_buffers down.  I recommend trying 10GB or so to start. 

>
> --Richard
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance


pgsql-performance by date:

Previous
From: Scott Carey
Date:
Subject: Re: Block at a time ...
Next
From: Craig James
Date:
Subject: Re: why does swap not recover?