Re: Hardware advice - Mailing list pgsql-performance

On Fri, 30 May 2003, Roman Fail wrote:

> Based on what you've said, I would guess you are considering the Dell PowerEdge 2650 since it has 5 drive bays.  If
youcould afford the rackspace and just a bit more money, I'd get the tower configuration 2600 with 6 drive bays (and
rackrails if needed - Dell even gives you a special rackmount faceplate if you order a tower with rack rails).  This
wouldallow you to have this configuration, which I think would be about ideal for the price range you are looking at: 
>
> * Linux kernel RAID

Actually, I think he was looking at hardware RAID, but I was recommending
software RAID as at least an option.  I've found that on modern hardware
with late model kernels, Linux is pretty fast with straight RAID, but not
as good with layering it, fyi.  I haven't tested since 2.4.9 though, so
things may well have changed, and hopefully for the better, in relation to
running fast in layered RAID.

they both would likely work well, but going with a sub par HW raid card
will make the system slower than the kernel sw RAID.

> * Dual processors - better than a single faster processor, especially
> with concurrent user load and software RAID on top of that
> * 2x36GB in RAID-1 (for OS and WAL)
> * 4x146GB in RAID-10 (for data) (alternative: 4-disk RAID-5)
>
> The RAID-10 array gives you the same amount of space you would have
> with a 3-disk RAID-5 and improved fault tolerance.  Although I'm pretty
> sure your drives won't be hot-swappable with the software RAID - I've
> never actually had to do it.

I agree that 6 drives makes this a much better option.

Actually, the hot swappable issue can only be accomplished in linux kernel
sw raid by using multiple controllers.  It's not really "hot swap" because
you have to basically reset that card and it's information about which
drives are on it.  Using two controllers, where one runs one RAID0 set,
and the other runs another RAID0 set, and you run a RAID1 on top, you can
then use hot swap shoes and replace failed drives.

The improved fault tolerance of the RAID 1+0 is minimal over the RAID5 if
the RAID5 has a hot spare, but it is there.

I've removed and added drives to running arrays, and the raidhotadd
program to do it is quite easy to drive.  It all seemed to work quite
well.  The biggest problem you'll note when a drive fails is that the
kernel / scsi driver will keep resetting the bus and timing out the
device, so with a failed device, linux kernel RAID can be a bit doggish
until you restart the SCSI driver so it KNOWs the drive's not there and
quits asking for it over and over.

> I can't say I like Scott's idea much because the WAL and OS are
> competing for disk time with the data since they are on the same
> physical disk.  In a database that is mainly reads with few writes,
> this wouldn't be such a problem though.

You'd be surprised how often this is a non-issue.  If you're writing
20,000 records every 10 minutes or so, the location of the WAL file is not
that important.  The machine will lug for a few seconds, insert, and be
done.  The speed increase averaged out over time is almost nothing.

Now, transactional systems are a whole nother enchilada.  I got the
feeling from the original post this was more a batch processing kinda
thing.

I knew the solution I was giving was suboptimal on performance (I might
have even alluded to that...).  I was going more for maximizing use of
rack space and getting the most storage. I think the user said that this
project might well grow to 250 or 300 gig, so size is probably more or as
important as speed for this system.

RAID5 is pretty much the compromise RAID set.  It's not necessarily the
fastest, it certainly isn't the sexiest, but it provides a lot of storage
for very little redundancy cost, and with a hot spare it's pretty much
24/7 with a couple days off a year for scheduled maintenance.  Combine
that with having n-1 number of platters for each read to be spread across
make it a nice choice for data warehousing or report serving.

Whatever he does, he should make sure he turns off atime on the data
partition.  That can utterly kill a postgresql / linux box by a factor of
right at two for someone doing small reads.