Thread: opinion on RAID choice

opinion on RAID choice

From
Vivek Khera
Date:
I just ran a handful of tests on a 14-disk array on a SCSI hardware
RAID card.

From some quickie benchmarks using the bonnie++ benchmark, it appears
that the RAID5 across all 14 disks is a bit faster than RAID50 and
noticeably faster than RAID10...

Sample numbers for a 10Gb file (speed in Kbytes/second)

                   RAID5     RAID50       RAID10
sequential write:  39728     37568        23533
read/write file:   13831     13289        11400
sequential read:   52184     51529        54222


Hardware is a Dell 2650 dual Xeon, 4GB Ram, PERC3/DC RAID card with
14 external U320 SCSI 15kRPM drives.  Software is FreeBSD 4.8 with the
default newfs settings.

The RAID drives were configured with 32k stripe size.  From informal
tests it doesn't seem to make much difference in the bonnie++
benchmark to go with 64k stripe on the RAID10 (didn't test it with
RAID5 or RAID50).  They say use larger stripe size for sequential
access, and lower for random access.

My concern is speed.  Any RAID config on this system has more disk
space than I will need for a LOOONG time.

My Postgres load is a heavy mix of select/update/insert.  ie, it is a
very actively updated and read database.

The conventional wisdom has been to use RAID10, but with 14 disks, I'm
kinda leaning toward RAID50 or perhaps just RAID5.

Has anyone else done similar tests of different RAID levels?  What
were your conclusions?

Raw output from bonnie++ available upon request.

Re: opinion on RAID choice

From
"scott.marlowe"
Date:
On Thu, 28 Aug 2003, Vivek Khera wrote:

> I just ran a handful of tests on a 14-disk array on a SCSI hardware
> RAID card.

SNIP

> Has anyone else done similar tests of different RAID levels?  What
> were your conclusions?

Yes I have.  I had a 6 disk array plus 2 disks inside my machine (this was
on a Sparc 20 with 4 narrow SCSI channels and the disks spread across them
evenly, using RH6.2 and linux sw raid.

My results were about the same as yours, RAID1+0 tended to beat RAID5 at
reads, while RAID5 tended to win at writes.

There's an old wive's tale that RAID5 has to touch every single disk in a
stripe when writing, which simply isn't true.  I believe that many old
controllers (decades back, 286 land kinda stuff) might have done it this
way, and so people kept thinking this was how RAID5 worked, and avoided
it.

My experience has been that once you get past 6 disks, RAID5 is faster
than RAID1+0.


Re: opinion on RAID choice

From
Vivek Khera
Date:
>>>>> "sm" == scott marlowe <scott.marlowe> writes:

sm> My experience has been that once you get past 6 disks, RAID5 is faster
sm> than RAID1+0.

Any opinion on stripe size for the RAID?

Re: opinion on RAID choice

From
"scott.marlowe"
Date:
On Thu, 28 Aug 2003, Vivek Khera wrote:

> >>>>> "sm" == scott marlowe <scott.marlowe> writes:
>
> sm> My experience has been that once you get past 6 disks, RAID5 is faster
> sm> than RAID1+0.
>
> Any opinion on stripe size for the RAID?

That's more determined by what kind of data you're gonna be handling.  If
you want to do lots of little financial transactions, then 32k or less is
good.  If you're gonna store moderately large text fields and such, then
going above 32k or 64k is usually a good idea.


Re: opinion on RAID choice

From
Andrew Sullivan
Date:
On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
>
> My experience has been that once you get past 6 disks, RAID5 is faster
> than RAID1+0.

Also depends on your filesystem and volume manager.  As near as I can
tell, you do _not_ want to use RAID 5 with Veritas.

A

--
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110


Re: opinion on RAID choice

From
Larry Rosenman
Date:

--On Tuesday, September 02, 2003 12:14:34 -0400 Andrew Sullivan
<andrew@libertyrms.info> wrote:

> On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
>>
>> My experience has been that once you get past 6 disks, RAID5 is faster
>> than RAID1+0.
>
> Also depends on your filesystem and volume manager.  As near as I can
> tell, you do _not_ want to use RAID 5 with Veritas.
Out of curiosity, why?

I have Veritas Doc up (since UnixWare has it) at:

http://www.lerctr.org:8458/en/Navpages/FShome.html

if anyone wants to read.

LER

>
> A



--
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Re: opinion on RAID choice

From
Ron Johnson
Date:
On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote:
> On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
> >
> > My experience has been that once you get past 6 disks, RAID5 is faster
> > than RAID1+0.
>
> Also depends on your filesystem and volume manager.  As near as I can
> tell, you do _not_ want to use RAID 5 with Veritas.

Why should Veritas care?  Or is it that Veritas has a high overhead
of small block writes?

--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@cox.net
Jefferson, LA USA

"Millions of Chinese speak Chinese, and it's not hereditary..."
Dr. Dean Edell


Re: opinion on RAID choice

From
Andrew Sullivan
Date:
On Tue, Sep 02, 2003 at 11:24:16AM -0500, Larry Rosenman wrote:
> >tell, you do _not_ want to use RAID 5 with Veritas.
> Out of curiosity, why?

What I keep hearing through various back channels is that, if you pay
folks from Veritas to look at your installation, and they see RAID 5,
they suggest you move it to 1+0.  I haven't any idea why.  It could
be just a matter of preference; it could be prejudice; it could be
baseless faith in the inefficiency of RAID5; it could be they have
stock in a drive company; or it could be they know about some strange
bug that they haven't an idea how to fix.  Nobody's ever been
able/willing to tell me.

A

----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110


Re: opinion on RAID choice

From
Greg Spiegelberg
Date:
Ron Johnson wrote:
> On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote:
>
>>On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
>>
>>>My experience has been that once you get past 6 disks, RAID5 is faster
>>>than RAID1+0.
>>
>>Also depends on your filesystem and volume manager.  As near as I can
>>tell, you do _not_ want to use RAID 5 with Veritas.
>
>
> Why should Veritas care?  Or is it that Veritas has a high overhead
> of small block writes?
>


I agree with Scott however only when it's hardware RAID 5 and only
certain hardware implementations of it.  A Sun A1000 RAID 5 is not
equal to a Sun T3.  Putting disk technologies aside, the A1000 array
XOR function is in software whereas the T3 is implemented in hardware.
Additionally, most external hardware based RAID systems have some
form of battery backup to ensure all data is written.

Veritas Volume Manager and even Linux, HP-UX and AIX LVM works just
fine when slicing & dicing but not for stitching LUN's together.  IMHO,
if you have the $$ for VxVM buy a hardware based RAID solution as well
and let it do the work.

Greg

--
Greg Spiegelberg
  Sr. Product Development Engineer
  Cranel, Incorporated.
  Phone: 614.318.4314
  Fax:   614.431.8388
  Email: gspiegelberg@Cranel.com
Cranel. Technology. Integrity. Focus.



Re: opinion on RAID choice

From
Ron Johnson
Date:
On Tue, 2003-09-02 at 11:47, Greg Spiegelberg wrote:
> Ron Johnson wrote:
> > On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote:
> >
> >>On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
> >>
> >>>My experience has been that once you get past 6 disks, RAID5 is faster
> >>>than RAID1+0.
> >>
> >>Also depends on your filesystem and volume manager.  As near as I can
> >>tell, you do _not_ want to use RAID 5 with Veritas.
> >
> >
> > Why should Veritas care?  Or is it that Veritas has a high overhead
> > of small block writes?
> >
>
>
> I agree with Scott however only when it's hardware RAID 5 and only
> certain hardware implementations of it.  A Sun A1000 RAID 5 is not
> equal to a Sun T3.  Putting disk technologies aside, the A1000 array
> XOR function is in software whereas the T3 is implemented in hardware.
> Additionally, most external hardware based RAID systems have some
> form of battery backup to ensure all data is written.
>
> Veritas Volume Manager and even Linux, HP-UX and AIX LVM works just
> fine when slicing & dicing but not for stitching LUN's together.  IMHO,
> if you have the $$ for VxVM buy a hardware based RAID solution as well
> and let it do the work.

Ah, shows how isolated, or behind the times, I am.  I thought that
Veritas just handled backups.  Never cared about looking at it to
do anything, else we always use h/w RAID storage controllers.

--
-----------------------------------------------------------------
Ron Johnson, Jr. ron.l.johnson@cox.net
Jefferson, LA USA

After listening to many White House, Pentagon & CENTCOM
briefings in both Gulf Wars, it is my firm belief that most
"senior correspondents" either have serious agendas that don't
get shaken by facts, or are dumb as dog feces.