Re: hanging for 30sec when checkpointing - Mailing list pgsql-admin

From Steve Crawford
Subject Re: hanging for 30sec when checkpointing
Date
Msg-id 200402121457.59468.scrawford@pinpointresearch.com
Whole thread Raw
In response to Re: hanging for 30sec when checkpointing  (gjm@caledoncard.com (Greg Mennie))
Responses Re: hanging for 30sec when checkpointing  ("scott.marlowe" <scott.marlowe@ihs.com>)
Re: hanging for 30sec when checkpointing  (Greg Mennie <gjm@caledoncard.com>)
List pgsql-admin
> > I'm running a reasonable sized (~30Gb) 7.3.4 database on Linux
> > and I'm getting some weird performance at times.
<snip>
> I am having a similar problem and this is what I've found so far:
>
> During the checkpoint the volume of data that's written isn't very
> high and it goes on for a fairly long time (up to 20 seconds) at a
> rate that appears to be well below our disk array's potential.  The
> volume of data written is usually 1-5 MB/sec on an array that we've
> tested to sustain over 50 MB/sec  (sequential writes, of course).
>
> It turns out that what's going on is that the command queue for the
> RAID array (3Ware RAID card) is filling up during the checkpoint
> and is staying at the max (254 commands) for most of the
> checkpoint.  The odd lucky insert appears to work, but is extremely
> slow.  In our case, the WAL files are on the same array as the data
> files, so everything grinds to a halt.

I spoke with some 3Ware reps at a trade show and they recommended
adding the following to /etc/sysctl.conf:
vm.max-readahead = 256
vm.min-readahead = 128

These settings take effect at boot. To change on a running system:
echo 256 > /proc/sys/vm/max-readahead
echo 128 > /proc/sys/vm/min-readahead

This advice was specific to the 3Ware card on Linux.

Cheers,
Steve


pgsql-admin by date:

Previous
From: Andrew Sullivan
Date:
Subject: Re: reduce downtime when upgrading 7.3 -> 7.4
Next
From: Godshall Michael
Date:
Subject: Re: Dumping from version 7.3.4 to 7.4.1