Thread: opinion on RAID choice
I just ran a handful of tests on a 14-disk array on a SCSI hardware
RAID card.
From some quickie benchmarks using the bonnie++ benchmark, it appears
that the RAID5 across all 14 disks is a bit faster than RAID50 and
noticeably faster than RAID10...
Sample numbers for a 10Gb file (speed in Kbytes/second)
RAID5 RAID50 RAID10
sequential write: 39728 37568 23533
read/write file: 13831 13289 11400
sequential read: 52184 51529 54222
Hardware is a Dell 2650 dual Xeon, 4GB Ram, PERC3/DC RAID card with
14 external U320 SCSI 15kRPM drives. Software is FreeBSD 4.8 with the
default newfs settings.
The RAID drives were configured with 32k stripe size. From informal
tests it doesn't seem to make much difference in the bonnie++
benchmark to go with 64k stripe on the RAID10 (didn't test it with
RAID5 or RAID50). They say use larger stripe size for sequential
access, and lower for random access.
My concern is speed. Any RAID config on this system has more disk
space than I will need for a LOOONG time.
My Postgres load is a heavy mix of select/update/insert. ie, it is a
very actively updated and read database.
The conventional wisdom has been to use RAID10, but with 14 disks, I'm
kinda leaning toward RAID50 or perhaps just RAID5.
Has anyone else done similar tests of different RAID levels? What
were your conclusions?
Raw output from bonnie++ available upon request.
On Thu, 28 Aug 2003, Vivek Khera wrote: > I just ran a handful of tests on a 14-disk array on a SCSI hardware > RAID card. SNIP > Has anyone else done similar tests of different RAID levels? What > were your conclusions? Yes I have. I had a 6 disk array plus 2 disks inside my machine (this was on a Sparc 20 with 4 narrow SCSI channels and the disks spread across them evenly, using RH6.2 and linux sw raid. My results were about the same as yours, RAID1+0 tended to beat RAID5 at reads, while RAID5 tended to win at writes. There's an old wive's tale that RAID5 has to touch every single disk in a stripe when writing, which simply isn't true. I believe that many old controllers (decades back, 286 land kinda stuff) might have done it this way, and so people kept thinking this was how RAID5 worked, and avoided it. My experience has been that once you get past 6 disks, RAID5 is faster than RAID1+0.
>>>>> "sm" == scott marlowe <scott.marlowe> writes: sm> My experience has been that once you get past 6 disks, RAID5 is faster sm> than RAID1+0. Any opinion on stripe size for the RAID?
On Thu, 28 Aug 2003, Vivek Khera wrote: > >>>>> "sm" == scott marlowe <scott.marlowe> writes: > > sm> My experience has been that once you get past 6 disks, RAID5 is faster > sm> than RAID1+0. > > Any opinion on stripe size for the RAID? That's more determined by what kind of data you're gonna be handling. If you want to do lots of little financial transactions, then 32k or less is good. If you're gonna store moderately large text fields and such, then going above 32k or 64k is usually a good idea.
On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote:
>
> My experience has been that once you get past 6 disks, RAID5 is faster
> than RAID1+0.
Also depends on your filesystem and volume manager. As near as I can
tell, you do _not_ want to use RAID 5 with Veritas.
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
--On Tuesday, September 02, 2003 12:14:34 -0400 Andrew Sullivan <andrew@libertyrms.info> wrote: > On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote: >> >> My experience has been that once you get past 6 disks, RAID5 is faster >> than RAID1+0. > > Also depends on your filesystem and volume manager. As near as I can > tell, you do _not_ want to use RAID 5 with Veritas. Out of curiosity, why? I have Veritas Doc up (since UnixWare has it) at: http://www.lerctr.org:8458/en/Navpages/FShome.html if anyone wants to read. LER > > A -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote: > On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote: > > > > My experience has been that once you get past 6 disks, RAID5 is faster > > than RAID1+0. > > Also depends on your filesystem and volume manager. As near as I can > tell, you do _not_ want to use RAID 5 with Veritas. Why should Veritas care? Or is it that Veritas has a high overhead of small block writes? -- ----------------------------------------------------------------- Ron Johnson, Jr. ron.l.johnson@cox.net Jefferson, LA USA "Millions of Chinese speak Chinese, and it's not hereditary..." Dr. Dean Edell
On Tue, Sep 02, 2003 at 11:24:16AM -0500, Larry Rosenman wrote:
> >tell, you do _not_ want to use RAID 5 with Veritas.
> Out of curiosity, why?
What I keep hearing through various back channels is that, if you pay
folks from Veritas to look at your installation, and they see RAID 5,
they suggest you move it to 1+0. I haven't any idea why. It could
be just a matter of preference; it could be prejudice; it could be
baseless faith in the inefficiency of RAID5; it could be they have
stock in a drive company; or it could be they know about some strange
bug that they haven't an idea how to fix. Nobody's ever been
able/willing to tell me.
A
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
Ron Johnson wrote: > On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote: > >>On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote: >> >>>My experience has been that once you get past 6 disks, RAID5 is faster >>>than RAID1+0. >> >>Also depends on your filesystem and volume manager. As near as I can >>tell, you do _not_ want to use RAID 5 with Veritas. > > > Why should Veritas care? Or is it that Veritas has a high overhead > of small block writes? > I agree with Scott however only when it's hardware RAID 5 and only certain hardware implementations of it. A Sun A1000 RAID 5 is not equal to a Sun T3. Putting disk technologies aside, the A1000 array XOR function is in software whereas the T3 is implemented in hardware. Additionally, most external hardware based RAID systems have some form of battery backup to ensure all data is written. Veritas Volume Manager and even Linux, HP-UX and AIX LVM works just fine when slicing & dicing but not for stitching LUN's together. IMHO, if you have the $$ for VxVM buy a hardware based RAID solution as well and let it do the work. Greg -- Greg Spiegelberg Sr. Product Development Engineer Cranel, Incorporated. Phone: 614.318.4314 Fax: 614.431.8388 Email: gspiegelberg@Cranel.com Cranel. Technology. Integrity. Focus.
On Tue, 2003-09-02 at 11:47, Greg Spiegelberg wrote: > Ron Johnson wrote: > > On Tue, 2003-09-02 at 11:14, Andrew Sullivan wrote: > > > >>On Thu, Aug 28, 2003 at 03:26:14PM -0600, scott.marlowe wrote: > >> > >>>My experience has been that once you get past 6 disks, RAID5 is faster > >>>than RAID1+0. > >> > >>Also depends on your filesystem and volume manager. As near as I can > >>tell, you do _not_ want to use RAID 5 with Veritas. > > > > > > Why should Veritas care? Or is it that Veritas has a high overhead > > of small block writes? > > > > > I agree with Scott however only when it's hardware RAID 5 and only > certain hardware implementations of it. A Sun A1000 RAID 5 is not > equal to a Sun T3. Putting disk technologies aside, the A1000 array > XOR function is in software whereas the T3 is implemented in hardware. > Additionally, most external hardware based RAID systems have some > form of battery backup to ensure all data is written. > > Veritas Volume Manager and even Linux, HP-UX and AIX LVM works just > fine when slicing & dicing but not for stitching LUN's together. IMHO, > if you have the $$ for VxVM buy a hardware based RAID solution as well > and let it do the work. Ah, shows how isolated, or behind the times, I am. I thought that Veritas just handled backups. Never cared about looking at it to do anything, else we always use h/w RAID storage controllers. -- ----------------------------------------------------------------- Ron Johnson, Jr. ron.l.johnson@cox.net Jefferson, LA USA After listening to many White House, Pentagon & CENTCOM briefings in both Gulf Wars, it is my firm belief that most "senior correspondents" either have serious agendas that don't get shaken by facts, or are dumb as dog feces.