Thread: RAID 5 and postgresql

On Wed, 2006-01-18 at 10:46, Hrishikesh Deshmukh wrote:
> Hi All,
>
> Are there any pointers for RAID tuning (RAID5) with PostgreSQL 8.0?!

There's a bi monthly conversation on this subject here and in the
perform list.  I'd recommend searching the perform list on the subject.
Here's the short version:

1:  Get the best RAID card you can afford.  Battery backed cache is
important for good write performance.

2:  In many circumstances, RAID 10 is a much better choice than RAID5.
However, some RAID cards are notoriously bad at layering RAID levels.
So is the linux kernel, unless some one fixed that.  The solution to
this problem is often to run one layer in hardware, and the other in the
OS.  I.e. take 10 drives.  Build 5 mirror sets on the hardware
controller.  Build a RAID 0 strip set out of them in the linux kernel
layer.

3:  RAID5 is a decent choice for a reporting / mostly read database.

4:  Lately, the Areca cards seem to be getting a lot of good marks.  The
LSI and Escalade cards are pretty good.  I've had horrible luck with
Adaptec, but some folks have had better luck than me.

5:  For RAID 5 to perform well with lots of writes, you need plenty of
drives and that battery backed cache mentioned above.  With fewer than 8
or 10 drives, the RAID 10 will always be much faster.

6:  Drives are cheap, data ain't, so you may be better off with RAID
10.  At least have a look at it.

I would suppliment this with just saying that your controller card is
your performance,
the only cards I've seen score well on linux, and people have
expressed on this list for SCSI are the LSI card, for SATA, LSI, 3ware
(now AMCC) and Areca claim good linux support and seem to work well.
Steer full clear of Adaptec, Dell and Compaq controllers, and their
linux support is abysmal, and the performance reflects that,
particularly in RAID 5.

Anyone have any experience with the IBM controllers who might comment
on those?  Are there any names I've missed?

Alex.

On 1/18/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
> On Wed, 2006-01-18 at 10:46, Hrishikesh Deshmukh wrote:
> > Hi All,
> >
> > Are there any pointers for RAID tuning (RAID5) with PostgreSQL 8.0?!
>
> There's a bi monthly conversation on this subject here and in the
> perform list.  I'd recommend searching the perform list on the subject.
> Here's the short version:
>
> 1:  Get the best RAID card you can afford.  Battery backed cache is
> important for good write performance.
>
> 2:  In many circumstances, RAID 10 is a much better choice than RAID5.
> However, some RAID cards are notoriously bad at layering RAID levels.
> So is the linux kernel, unless some one fixed that.  The solution to
> this problem is often to run one layer in hardware, and the other in the
> OS.  I.e. take 10 drives.  Build 5 mirror sets on the hardware
> controller.  Build a RAID 0 strip set out of them in the linux kernel
> layer.
>
> 3:  RAID5 is a decent choice for a reporting / mostly read database.
>
> 4:  Lately, the Areca cards seem to be getting a lot of good marks.  The
> LSI and Escalade cards are pretty good.  I've had horrible luck with
> Adaptec, but some folks have had better luck than me.
>
> 5:  For RAID 5 to perform well with lots of writes, you need plenty of
> drives and that battery backed cache mentioned above.  With fewer than 8
> or 10 drives, the RAID 10 will always be much faster.
>
> 6:  Drives are cheap, data ain't, so you may be better off with RAID
> 10.  At least have a look at it.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>                http://archives.postgresql.org
>

Hi,

> I would suppliment this with just saying that your controller card is
> your performance,
> the only cards I've seen score well on linux, and people have
> expressed on this list for SCSI are the LSI card, for SATA, LSI, 3ware
> (now AMCC) and Areca claim good linux support and seem to work well.
> Steer full clear of Adaptec, Dell and Compaq controllers, and their
> linux support is abysmal, and the performance reflects that,
> particularly in RAID 5.

Dell has used (and rebranded) Adaptec and LSI controllers for their PERC
series, and I agree that the Adaptec controllers perform badly. As far as I
know the LSI based controllers are quite good (and some come with 256MB
battery backed cache, which is nice :-)

Sander.

On Sat, 2006-01-21 at 07:09, Sander Steffann wrote:
> Hi,
>
> > I would suppliment this with just saying that your controller card is
> > your performance,
> > the only cards I've seen score well on linux, and people have
> > expressed on this list for SCSI are the LSI card, for SATA, LSI, 3ware
> > (now AMCC) and Areca claim good linux support and seem to work well.
> > Steer full clear of Adaptec, Dell and Compaq controllers, and their
> > linux support is abysmal, and the performance reflects that,
> > particularly in RAID 5.
>
> Dell has used (and rebranded) Adaptec and LSI controllers for their PERC
> series, and I agree that the Adaptec controllers perform badly. As far as I
> know the LSI based controllers are quite good (and some come with 256MB
> battery backed cache, which is nice :-)

Last place I worked we used Dell rackmounts (2600 series mostly) and
they came, by default with the Adaptec based controllers.  Those were
horrific, locking up under load, really poor performance, etc...

The LSIs, as you mentioned, were much better.  We had exactly one Dell
2600 with the LSI (hmmm.  Bet you can't guess who specced that machine
out, eh?  hehe) with 256 Meg BBCache.  While it never locked up or hung,
it's I/O was noticeable slower than the machine it replaced, which also
had an LSI RAID controller with BBCache, bascially, the same chipset.

I'm not sure if it's Dell's BIOS on the mobos, or something with the LSI
cards, but the performance was substandard.

So if you're working somewhere that you simply have to use Dell (not
uncommon), at least make sure you get the LSI based RAID controller.

At 10:01 AM 1/23/2006 -0600, Scott Marlowe wrote:

>I'm not sure if it's Dell's BIOS on the mobos, or something with the LSI
>cards, but the performance was substandard.
>
>So if you're working somewhere that you simply have to use Dell (not
>uncommon), at least make sure you get the LSI based RAID controller.

How about software RAID?

Linux software RAID appears to perform better than most RAID controllers
except perhaps those that can do read interleaving for RAID1 (I believe
some 3ware controllers can do it). Linux RAID mirroring doesn't do read
interleaving, only read balancing, which may not be so good for a single
sequential read, but pretty good for concurrent sequential reads - each
drive in a mirror set can handle one sequential read.

I find many of these RAID controllers fail significantly more than basic
SCSI controllers (which hardly ever fail). And the support under Linux for
such controllers can be a bit patchy sometimes - you want to be able to
easily know if a drive has died.

It just seems strange to pay a fair bit for something that doesn't perform
well and is less reliable.

Of course you get the "convenience" of the RAID stuff being abstracted away
so it just looks like one drive.

On Jan 21, 2006, at 8:09 AM, Sander Steffann wrote:

> Dell has used (and rebranded) Adaptec and LSI controllers for their
> PERC series, and I agree that the Adaptec controllers perform
> badly. As far as I know the LSI based controllers are quite good
> (and some come with 256MB battery backed cache, which is nice :-)

Over the last 5 years I've had a *LOT* of Dell equipment, most of
which has RAID.   The adaptec-based ones are OK.  Not great
performance, but very easy to manage.

The LSI based ones are pretty good, but the older ones were somehow
deficient in that the expected performance was never achieved.

However, the latest one I have is a PE 1850 with the PERC43/Si (LSI
based) which is surprisingly fast.  It comes with battery + 256MB RAM
and just screams in RAID1 mode (only have two disks on this box).

I'd buy that server again anyday.

However, for my big DB server arrays, I always go with the LSI
branded cards and run them in RAID10 config.

Hi,

> How about software RAID?
>
> Linux software RAID appears to perform better than most RAID
> controllers except perhaps those that can do read interleaving
> for RAID1 (I believe some 3ware controllers can do it). Linux
> RAID mirroring doesn't do read interleaving, only read
> balancing, which may not be so good for a single sequential
> read, but pretty good for concurrent sequential reads - each
> drive in a mirror set can handle one sequential read.

Don't forget the battery backed cache for write performance. And because the
controller doesn't know about the RAID array booting can become a problem
when your first drive breaks.

> I find many of these RAID controllers fail significantly more
> than basic SCSI controllers (which hardly ever fail). And the
> support under Linux for such controllers can be a bit patchy
> sometimes - you want to be able to easily know if a drive has
> died.
>
> It just seems strange to pay a fair bit for something that
> doesn't perform well and is less reliable.

The Dell OpenManage tools can help you with that, and if the controller
fails (never happened yet here) you can just call Dell support, and within 4
hours they bring you a new one. (I had some strange problems with a server,
and Dell replaced the mainboard and memory very quickly)

I still choose the Dell LSI-based PERC4/Di where possible.
- Sander