Thread: Hardware related question: 3ware 9500S

Hi,

i don't know much about hard disks and raid controllers but often there is
some discussion about which raid controller rocks and which sucks. my hosting
company offers me a raid 10 with 4 serial-ata disks. They will use a "3ware
4-Port-RAID-Controller 9500S"

More than 4 disks are not possible. Most operations and all time-critical
operations are read-only using a lot of indices. My partioning plans are like
this:

disk 1: OS, tablespace
disk 2: indices, WAL, Logfiles

- Does my partitioning make sense?
- I want to know if 3ware 9500 S is recommended or if its one of those
controllers which sucks.

kind regards,
janning

On 4/12/06, Janning Vygen <vygen@gmx.de> wrote:
> Hi,
> disk 1: OS, tablespace
> disk 2: indices, WAL, Logfiles
> - Does my partitioning make sense?

with raid 10 all four drives will appear as a single physical device
shared by all.  I'm personally not a big fan of logical partitioning
of a single raid device unless you are trying to keep a physical
volume under 1 TB for example.  Each sync on the volume is guaranteed
to sync all 4 disks regardless of how you set your partitions up.

> - I want to know if 3ware 9500 S is recommended or if its one of those
> controllers which sucks.

escalade is a fairly full featured raid controller for the price.
consider it the ford taurus of raid controllers, it's functional and
practical but not sexy.  Their S line is not native sata but operates
over a pata->sata bridge.  Stay away from raid 5.

merlin

Thanks for your fast reply.

Am Mittwoch, 12. April 2006 18:31 schrieb Merlin Moncure:
> On 4/12/06, Janning Vygen <vygen@gmx.de> wrote:
> > Hi,
> > disk 1: OS, tablespace
> > disk 2: indices, WAL, Logfiles
> > - Does my partitioning make sense?
>
> with raid 10 all four drives will appear as a single physical device
> shared by all.  I'm personally not a big fan of logical partitioning
> of a single raid device unless you are trying to keep a physical
> volume under 1 TB for example.  Each sync on the volume is guaranteed
> to sync all 4 disks regardless of how you set your partitions up.

Ok, i am not a raid expert. but in my understanding RAID 10 is faster than two
RAID 1 arrays, aren't they? So, given that i can put up to 4 S-ATA disk in my
server and the mentioned raid controller. Would you prefer no-raid, RAID1 or
RAID 10?

> > - I want to know if 3ware 9500 S is recommended or if its one of those
> > controllers which sucks.
>
> escalade is a fairly full featured raid controller for the price.
> consider it the ford taurus of raid controllers, it's functional and
> practical but not sexy.  Their S line is not native sata but operates
> over a pata->sata bridge.  Stay away from raid 5.

thanks for your recommendation. ford taurus is ok for me :-)

kind regrads
janning

> ----- Original Message -----
> From: "Merlin Moncure" <mmoncure@gmail.com>
> To: "Janning Vygen" <vygen@gmx.de>
> Cc: <pgsql-general@postgresql.org>
> Sent: Wednesday, April 12, 2006 12:31 PM
> Subject: Re: [GENERAL] Hardware related question: 3ware 9500S
> [snip]

> > - I want to know if 3ware 9500 S is recommended or if its one of those
> > controllers which sucks.
>
> escalade is a fairly full featured raid controller for the price.
> consider it the ford taurus of raid controllers, it's functional and
> practical but not sexy.  Their S line is not native sata but operates
> over a pata->sata bridge.  Stay away from raid 5.
>
Hi Merlin

Why?  What's wrong with raid 5? I could well be wrong (given how little
attention I have paid to hardware over the past few years because of a focus
on developing software), but I was under the impression that of the raid
options available, raid 5 with hot swappable drives provided good data
protection and performance at a reasonably low cost.  Is the problem with
the concept of raid 5, or the common implementations?

Do you have a recommendation regarding whether the raid array is built into
the server running the RDBMS (in our case PostgreSQL), or located in a
network appliance dedicated to storing the data managed by the RDBMS?  If
you were asked to design a subnet that provides the best possible
performance and protection of the data, but without gold-plating anything,
what would you do?  How much redundancy would you build in, and at what
granularity?

Ted

On Wed, 2006-04-12 at 13:10, Ted Byers wrote:
> > ----- Original Message -----
> > From: "Merlin Moncure" <mmoncure@gmail.com>
> > To: "Janning Vygen" <vygen@gmx.de>
> > Cc: <pgsql-general@postgresql.org>
> > Sent: Wednesday, April 12, 2006 12:31 PM
> > Subject: Re: [GENERAL] Hardware related question: 3ware 9500S
> > [snip]
>
> > > - I want to know if 3ware 9500 S is recommended or if its one of those
> > > controllers which sucks.
> >
> > escalade is a fairly full featured raid controller for the price.
> > consider it the ford taurus of raid controllers, it's functional and
> > practical but not sexy.  Their S line is not native sata but operates
> > over a pata->sata bridge.  Stay away from raid 5.
> >
> Hi Merlin
>
> Why?  What's wrong with raid 5? I could well be wrong (given how little
> attention I have paid to hardware over the past few years because of a focus
> on developing software), but I was under the impression that of the raid
> options available, raid 5 with hot swappable drives provided good data
> protection and performance at a reasonably low cost.  Is the problem with
> the concept of raid 5, or the common implementations?
>
> Do you have a recommendation regarding whether the raid array is built into
> the server running the RDBMS (in our case PostgreSQL), or located in a
> network appliance dedicated to storing the data managed by the RDBMS?  If
> you were asked to design a subnet that provides the best possible
> performance and protection of the data, but without gold-plating anything,
> what would you do?  How much redundancy would you build in, and at what
> granularity?

There have been NUMEROUS discussions of RAID-5 versus RAID 1+0 in the
perform group in the last year or two.  Short version:

RAID 5 is useful, with large numbers of drives, for OLAP type databases,
where you're trying to get as much storage as possible from your
drives.  RAID 5 pretty much REQUIRES battery backed cache for decent
write performance, and even then, will saturate faster than RAID 1+0.
RAID-5 cannot survive multiple simultaneous drive failures.

RAID 1+0 requires better than average controllers, since many serialize
and lockstep data through the various layers of RAID on them.  It
provides less storage for a given number of drives.  It is faster for
OLTP workloads than RAID-5.  RAID 1+0 can survive multiple drive
failures as long as two drives in the same mirror set do not fail at
once.

With increasing number of drives, the chances of a RAID 5 failing go up
linearly, while the chances of RAID 1+0 failing due to multiple drive
failure stay the same.

----- Original Message -----
From: "Scott Marlowe" <smarlowe@g2switchworks.com>
To: "Ted Byers" <r.ted.byers@rogers.com>
Cc: "Merlin Moncure" <mmoncure@gmail.com>; "Janning Vygen" <vygen@gmx.de>;
"pgsql general" <pgsql-general@postgresql.org>
Sent: Wednesday, April 12, 2006 2:24 PM
Subject: Re: [GENERAL] Hardware related question: 3ware 9500S


> On Wed, 2006-04-12 at 13:10, Ted Byers wrote:
>> > ----- Original Message -----
>> > From: "Merlin Moncure" <mmoncure@gmail.com>
>> > To: "Janning Vygen" <vygen@gmx.de>
>> > Cc: <pgsql-general@postgresql.org>
>> > Sent: Wednesday, April 12, 2006 12:31 PM
>> > Subject: Re: [GENERAL] Hardware related question: 3ware 9500S
>> > [snip]
>>
>> Why?  What's wrong with raid 5? I could well be wrong (given how little
>> attention I have paid to hardware over the past few years because of a
>> focus
>> on developing software), but I was under the impression that of the raid
>> options available, raid 5 with hot swappable drives provided good data
>> protection and performance at a reasonably low cost.  Is the problem with
>> the concept of raid 5, or the common implementations?
>>
>> Do you have a recommendation regarding whether the raid array is built
>> into
>> the server running the RDBMS (in our case PostgreSQL), or located in a
>> network appliance dedicated to storing the data managed by the RDBMS?  If
>> you were asked to design a subnet that provides the best possible
>> performance and protection of the data, but without gold-plating
>> anything,
>> what would you do?  How much redundancy would you build in, and at what
>> granularity?
>
> There have been NUMEROUS discussions of RAID-5 versus RAID 1+0 in the
> perform group in the last year or two.  Short version:
>
Interesting.

I take it that "RAID 1+0" refers to a combination of Raid 1 and RAID 0.
What about RAID 10?  I am curious because RAID 10 has come out since the
last time I took a look at RAID technology.  I am not sure what it actually
does differently from RAID 5.

This question of data security is becoming of increasing importance to me
professionally since I will soon have to advise the company I'm working with
regarding how best to secure the data managed by the applications I'm
developing for them.  I will need overall guidelines to produce a design
that makes it virtually impossible for them to lose even on field in one
record.  The data is both sensitive and vital.  Fortunately, I have a few
months before we need to commit to anything.  Also, fortunately, with one
exception, the applications rely on a data feed that comes in once a day
after normal working hours, so I won't have to worry about writes to the DB
other than what my script does to load the datafeed into the DB.  All other
access is read only.  This should make it easier to produce a strategy to
protect the data from any kind of technology failure (software or hardware).
Cost is a factor, but reliability is much much more important!

Thanks,

Ted

On Wed, Apr 12, 2006 at 02:53:01PM -0400, Ted Byers wrote:
> I take it that "RAID 1+0" refers to a combination of Raid 1 and RAID 0.
> What about RAID 10?  I am curious because RAID 10 has come out since the
> last time I took a look at RAID technology.  I am not sure what it actually
> does differently from RAID 5.

AIUI, RAID 10 = RAID 1+0. Lame, I know. Similarly, some people have
invented RAID 50 = RAID 5+0.

Not sure if that's the official definition though, but that's the way
I've seen it used.
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Martijn van Oosterhout wrote:
> On Wed, Apr 12, 2006 at 02:53:01PM -0400, Ted Byers wrote:
>> I take it that "RAID 1+0" refers to a combination of Raid 1 and RAID 0.
>> What about RAID 10?  I am curious because RAID 10 has come out since the
>> last time I took a look at RAID technology.  I am not sure what it actually
>> does differently from RAID 5.
>
> AIUI, RAID 10 = RAID 1+0. Lame, I know. Similarly, some people have
> invented RAID 50 = RAID 5+0.
>
> Not sure if that's the official definition though, but that's the way
> I've seen it used.

Useful info on RAID definitions:

http://tinyurl.com/zhnmc

--
Until later, Geoffrey

Any society that would give up a little liberty to gain a little
security will deserve neither and lose both.  - Benjamin Franklin

On 4/12/06, Ted Byers <r.ted.byers@rogers.com> wrote:
>
> > ----- Original Message -----
> > From: "Merlin Moncure" <mmoncure@gmail.com>
> > To: "Janning Vygen" <vygen@gmx.de>
> > Cc: <pgsql-general@postgresql.org>
> > Sent: Wednesday, April 12, 2006 12:31 PM
> > Subject: Re: [GENERAL] Hardware related question: 3ware 9500S
> > [snip]
>
> > > - I want to know if 3ware 9500 S is recommended or if its one of those
> > > controllers which sucks.
> >
> > escalade is a fairly full featured raid controller for the price.
> > consider it the ford taurus of raid controllers, it's functional and
> > practical but not sexy.  Their S line is not native sata but operates
> > over a pata->sata bridge.  Stay away from raid 5.
> >
> Hi Merlin
>
> Why?  What's wrong with raid 5? I could well be wrong (given how little

there are reasons to go with raid 5 or other raids. where I work we
often do 14 drive raid 6 plus 1 hot swap on a 15 drive tray.  However,
for 4 drive raid, I think 0+1 is the by far the best choice.  For
three drive, I'd suggest two drive raid 1 plus hot swap.

> Do you have a recommendation regarding whether the raid array is built into
> the server running the RDBMS (in our case PostgreSQL), or located in a
> network appliance dedicated to storing the data managed by the RDBMS?  If
> you were asked to design a subnet that provides the best possible
> performance and protection of the data, but without gold-plating anything,
> what would you do?  How much redundancy would you build in, and at what
> granularity?

I would stay clear of cheaper NAS solutions (AoE, iscsi) unless you
really didn't care about performance.  In my experience the better
SANs are a good way to go if you need flexibility or easy managment
(especially if you need to do things besides database) without losing
performance.  A good SAN makes everything easy but boy do you pay for
it.

If you want most bang for the buck, I'd suggest either attached scsi
or sata (especially the latter).  With sata, 24 raptors will get you
insane performance for a very reasonable price.  Most of my apps are
cpu bound anyways.

On Wed, 2006-04-12 at 13:53, Ted Byers wrote:
> ----- Original Message -----
> From: "Scott Marlowe" <smarlowe@g2switchworks.com>
> >
> > There have been NUMEROUS discussions of RAID-5 versus RAID 1+0 in the
> > perform group in the last year or two.  Short version:
> >
> Interesting.

SNIP

> This question of data security is becoming of increasing importance to me
> professionally since I will soon have to advise the company I'm working with
> regarding how best to secure the data managed by the applications I'm
> developing for them.  I will need overall guidelines to produce a design
> that makes it virtually impossible for them to lose even on field in one
> record.  The data is both sensitive and vital.  Fortunately, I have a few
> months before we need to commit to anything.  Also, fortunately, with one
> exception, the applications rely on a data feed that comes in once a day
> after normal working hours, so I won't have to worry about writes to the DB
> other than what my script does to load the datafeed into the DB.  All other
> access is read only.  This should make it easier to produce a strategy to
> protect the data from any kind of technology failure (software or hardware).
> Cost is a factor, but reliability is much much more important!

When you say reliability, I'm not sure your definition is my
definition.  Is is that the database MUST be up during business hours,
even if the updates that happen during the day can't go through?  Or
even if those updates get lost and have to be re-entered that's OK, as
long as the data entered by the batch file at night is available for
business processes.

There are a lot of ways to set this up, and each tool in the tool box
has its advantages and disadvantages.  PITR, Slony Replication, pgpool,
pgcluster, mammoth replicator, bizgress, bizgress MPP...

I'd guess that the data entered during the day is the most important.
If this is so, you could set up slony replication on those tables to a
backup machine so that should the primary suffer catastrophic failure
you still have the inputs.

RAID is great, but it's no replacement for replication and / or point in
time recovery.  More an augment.

Merlin Moncure writes:

> escalade is a fairly full featured raid controller for the price.
> consider it the ford taurus of raid controllers, it's functional and
> practical but not sexy.  Their S line is not native sata but operates
> over a pata->sata bridge.  Stay away from raid 5.

Do you know if their raid 5 is better in the new 9550SX?

Or is the "Stay away from raid 5" more of a general comment that this type
of raid is not good for DBs?

Merlin Moncure writes:

> there are reasons to go with raid 5 or other raids. where I work we
> often do 14 drive raid 6 plus 1 hot swap on a 15 drive tray.

Raid 5 is different from raid 6.... To say that there are times it's ok to
use RAID 5 and then say you use raid 6... well... doesn't really say
anything about raid 5.

Also, what controller are you using?
From what I gather, raid 6 is less common and fewer cards support it (areca
is one of them I believe).

Raid 5 on the 9550SX is supposed to be significantly better than the 9500 series.

I would be carefull of benchmarks listed out there.  For instance, whilst looking for supporting material, I came cross this gem:
http://www.gamepc.com/labs/print_content.asp?id=9550sx4lp&cookie%5Ftest=1

They claim the they used a Tyan Thunder K8WE motherboard, and installed the RAID controllers in a 64-bit 133MHz PCI-X slot.  This motherboard doesn't have any 64-bit 133Mhz PCI-X slots! ( http://www.tyan.com/products/html/tigerk8we_spec.html).
It's no wonder that the other raid controllers showed significantly less performance than the PCI-e card.

This review from tomshardware:
http://www.tomshardware.com/2005/10/31/sata_spells_trouble_for_scsi_raid/page13.html

Suggests that the 9550SX is at least competitive with the others.

I know I like the 3ware/AMCC cards because of their very good RAID 10 performance.  I'm not a big RAID 5 fan.  RAID 5 sufferes the read before write penalty problem that make RAID 5 writes very slow, particularly noticebale in OLTP applications.  RAID 10 will almost always offer better write perfomance.

I wish we could set up an organization to do benchmarks with pgbench on various different RAID controllers/drives and publish the results.  I know I would pay money for that.

Alex

Alex Turner wrote:
> Raid 5 on the 9550SX is supposed to be significantly better than the
> 9500 series.
>
> I would be carefull of benchmarks listed out there.  For instance,
> whilst looking for supporting material, I came cross this gem:
>
http://www.gamepc.com/labs/print_content.asp?id=9550sx4lp&cookie%5Ftest=
1
>
> They claim the they used a Tyan Thunder K8WE motherboard, and
> installed the RAID controllers in a 64-bit 133MHz PCI-X slot.  This
> motherboard doesn't have any 64-bit 133Mhz PCI-X slots! (
> http://www.tyan.com/products/html/tigerk8we_spec.html).
> It's no wonder that the other raid controllers showed significantly
> less performance than the PCI-e card.

You're looking at the wrong board.  They are talking about the Tyan
*Thunder*, which does indeed have 64-bit PCI-X.  You are looking at the
Tyan **Tiger**, which does not.

--
Guy Rouillier

Alex Turner writes:

> Suggests that the 9550SX is at least competitive with the others.

Thanks for the links.

> I know I like the 3ware/AMCC cards because of their very good RAID 10
> performance.

Raid 10 is what I used on my last server and likely what I will use on the
next.


> I wish we could set up an organization to do benchmarks with pgbench on
> various different RAID controllers/drives and publish the results.  I know
> I would pay money for that.

Yes. That would be truly very usefull, although it would be very time
consuming.

I have the time to do it, but not the $$s ;)

Alex

Ahhhh... good point.

Why oh why does tyan have two boards with the same prefix ;)!!!

Alex