Thread: Experience with HP Smart Array P400 and SATA drives?
Has anyone benchmarked this controller (PCIe/4x, 512 MB BBC)? We try to use it with 8x SATA 1TB drives in RAID-5 mode under Linux, and measure strange values. An individual drive is capable of delivering 91 MB/sec sequential read performance, and we get values ~102MB/sec out of a 8-drive RAID5, seems to be ridiculous slow. Write performance seems to be much better, ~300 MB /sec - seems ok to me. I guess I must be doing something wrong, I cannot believe that a 500 € controller is delivering such poor performance.
>>> Mario Weilguni <mweilguni@sime.com> wrote: > Has anyone benchmarked this controller (PCIe/4x, 512 MB BBC)? We try to > use it with 8x SATA 1TB drives in RAID-5 mode under Linux, and measure > strange values. An individual drive is capable of delivering 91 MB/sec > sequential read performance, and we get values ~102MB/sec out of a > 8-drive RAID5, seems to be ridiculous slow. Write performance seems to > be much better, ~300 MB /sec - seems ok to me. What's your stripe size? -Kevin
On Tue, Dec 2, 2008 at 2:22 AM, Mario Weilguni <mweilguni@sime.com> wrote: > Has anyone benchmarked this controller (PCIe/4x, 512 MB BBC)? We try to use > it with 8x SATA 1TB drives in RAID-5 mode under Linux, and measure strange > values. An individual drive is capable of delivering 91 MB/sec sequential > read performance, and we get values ~102MB/sec out of a 8-drive RAID5, seems > to be ridiculous slow. Write performance seems to be much better, ~300 MB > /sec - seems ok to me. > > I guess I must be doing something wrong, I cannot believe that a 500 € > controller is delivering such poor performance. A few suggestions... Try to find the latest driver for your card, try using the card as nothing but a caching controller and run your RAID on software in linux (or whatever IS you're on). Test a 2 drive RAID-0 to see what kind of performance increase you get. If you can't dd a big file off of a RAID-0 at about 2x the rate of a single drive then something IS wrong with it. Try RAID 10. Try RAID-1 sets on the controller and RAID 0 over that in software.
> >>> Mario Weilguni <mweilguni@sime.com> wrote: > > strange values. An individual drive is capable of delivering 91 > > MB/sec > > > sequential read performance, and we get values ~102MB/sec out of a > > 8-drive RAID5, seems to be ridiculous slow. What command are you using to test the reads? Some recommendations to try: 1) /sbin/blockdev --setra 2048 device (where device is the partition or LVM volume) 2) Use XFS, and make sure your stripe settings match the RAID. Having said that, 102MB/sec sounds really low for any modern controller with 8 drives, regardless of tuning or filesystem choice. -- Corporations will ingest natural resources and defecate garbage until all resources are depleted, debt can no longer be repaid and our money becomes worthless - Jay Hanson
On Fri, 5 Dec 2008, Alan Hodgson wrote: > 1) /sbin/blockdev --setra 2048 device (where device is the partition or > LVM volume) Normally, when I see write speed dramatically faster than write, it does mean that something about the read-ahead is set wrong. While I don't have one to check, it looks to me like the P400 does its own read-ahead though, which may be more effective to tweak than what Linux does. I'd suggest taking a look at the settings with HP's admin utility and see how it's set for that and for allocation of RAM to the read cache. I've seen some RAID5 configs that put too much caching on the write side by default to compensate for the deficiencies of that RAID level, and you have to push some of that back toward the read side to balance it out right. > 2) Use XFS, and make sure your stripe settings match the RAID. XFS has good performance, but I can't get over how many system failure corruption reports I hear about it. In any case, there's no reason this system shouldn't perform fine on ext3. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Fri, Dec 5, 2008 at 5:29 PM, Greg Smith <gsmith@gregsmith.com> wrote: > XFS has good performance, but I can't get over how many system failure > corruption reports I hear about it. In any case, there's no reason this > system shouldn't perform fine on ext3. For simple testing you can take the file system out of the equation by just using dd to / from the raw partition. I have a feeling this HP RAID controller is just a low end piece o crap is the problem. I seem to remember references to the P800 series controller being quite a bit better than this one. But that was a while back.
Scott Marlowe schrieb: > On Tue, Dec 2, 2008 at 2:22 AM, Mario Weilguni <mweilguni@sime.com> wrote: > >> Has anyone benchmarked this controller (PCIe/4x, 512 MB BBC)? We try to use >> it with 8x SATA 1TB drives in RAID-5 mode under Linux, and measure strange >> values. An individual drive is capable of delivering 91 MB/sec sequential >> read performance, and we get values ~102MB/sec out of a 8-drive RAID5, seems >> to be ridiculous slow. Write performance seems to be much better, ~300 MB >> /sec - seems ok to me. >> >> I guess I must be doing something wrong, I cannot believe that a 500 € >> controller is delivering such poor performance. >> > > A few suggestions... Try to find the latest driver for your card, try > using the card as nothing but a caching controller and run your RAID > on software in linux (or whatever IS you're on). Test a 2 drive > RAID-0 to see what kind of performance increase you get. If you can't > dd a big file off of a RAID-0 at about 2x the rate of a single drive > then something IS wrong with it. Try RAID 10. Try RAID-1 sets on the > controller and RAID 0 over that in software. > > I've already tried Softraid with individual drives, performs much better. However, it's no option to use softraid, so I'm stuck. The card has the latest firmware installed, and there are no drivers needed, they're already included in the linux kernel. I still think we must be doing something wrong here, I googled the controller and Linux, and did not find anything indicating a problem. The HP SmartArray series is quite common, so a lot of users would have the same problem. Thanks!
Kevin Grittner schrieb: >>>> Mario Weilguni <mweilguni@sime.com> wrote: >>>> >> Has anyone benchmarked this controller (PCIe/4x, 512 MB BBC)? We try >> > to > >> use it with 8x SATA 1TB drives in RAID-5 mode under Linux, and >> > measure > >> strange values. An individual drive is capable of delivering 91 >> > MB/sec > >> sequential read performance, and we get values ~102MB/sec out of a >> 8-drive RAID5, seems to be ridiculous slow. Write performance seems >> > to > >> be much better, ~300 MB /sec - seems ok to me. >> > > What's your stripe size? > > -Kevin > We used the default settings, it's 64k. Might a bigger value help here?
Alan Hodgson schrieb: >>>>> Mario Weilguni <mweilguni@sime.com> wrote: >>>>> >>> strange values. An individual drive is capable of delivering 91 >>> MB/sec >>> >>> sequential read performance, and we get values ~102MB/sec out of a >>> 8-drive RAID5, seems to be ridiculous slow. >>> > > > What command are you using to test the reads? > > Some recommendations to try: > > 1) /sbin/blockdev --setra 2048 device (where device is the partition or LVM > volume) > > 2) Use XFS, and make sure your stripe settings match the RAID. > > Having said that, 102MB/sec sounds really low for any modern controller with > 8 drives, regardless of tuning or filesystem choice. > > First, thanks alot for this and all the other answers. I measured the raw device performance: dd if=/dev/cciss/c0d0 bs=64k count=100000 of=/dev/null I get poor performance when all 8 drives are configured as one, large RAID-5, and slightly poorer performance when configured as JBOD. In production, we use XFS as FS, but I doubt this has anything to do with FS tuning.
On Tue, Dec 9, 2008 at 5:17 AM, Mario Weilguni <mweilguni@sime.com> wrote: > Alan Hodgson schrieb: >>>>>> >>>>>> Mario Weilguni <mweilguni@sime.com> wrote: >>>>>> >>>> >>>> strange values. An individual drive is capable of delivering 91 >>>> MB/sec >>>> sequential read performance, and we get values ~102MB/sec out of a >>>> 8-drive RAID5, seems to be ridiculous slow. >> >> >> What command are you using to test the reads? >> >> Some recommendations to try: >> >> 1) /sbin/blockdev --setra 2048 device (where device is the partition or >> LVM volume) >> >> 2) Use XFS, and make sure your stripe settings match the RAID. >> >> Having said that, 102MB/sec sounds really low for any modern controller >> with 8 drives, regardless of tuning or filesystem choice. >> >> > > First, thanks alot for this and all the other answers. > > I measured the raw device performance: > dd if=/dev/cciss/c0d0 bs=64k count=100000 of=/dev/null > > I get poor performance when all 8 drives are configured as one, large > RAID-5, and slightly poorer performance when configured as JBOD. In > production, we use XFS as FS, but I doubt this has anything to do with FS > tuning. Yeah, having just trawled the pgsql-performance archives, there are plenty of instances of people having terrible performance from HP smart array controllers before the P800. Is it possible for you to trade up to a better RAID controller? Whichever salesman sold you the P400 should take one for the team and make this right for you.
On Tue, 2008-12-09 at 13:10 +0100, Mario Weilguni wrote: > Scott Marlowe schrieb: > > On Tue, Dec 2, 2008 at 2:22 AM, Mario Weilguni <mweilguni@sime.com> wrote: > I still think we must be doing something wrong here, I googled the > controller and Linux, and did not find anything indicating a problem. > The HP SmartArray series is quite common, so a lot of users would have > the same problem. Yes the SmartArray series is quite common and actually know to perform reasonably well, in RAID 10. You still appear to be trying RAID 5. Joshua D. Drake > > Thanks! > -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
We reached a fairly good performance on a P400 controller (8 SATA 146GB 2,5" 10k rpm) with raid5 or raid6 Linux software raid: the writing bandwidth reached about 140 MB/s sustained throughput (the hardware raid5 gave a sustained 20 MB/s...). With a second, equal controller (16 disks) we reached (raid6 spanning all 16 disks) about 200 MB/s sustained. The CPU load is negligible. Reading performance is about 20% better. Best regards and my apologies for my bad English. GT P.S.: on a P800, 12 SATA 750GB 3,5" 7200 rpm, the hardware raid5 writing performance was about 30 MB/s, software raid5 is between 60 and 80 MB/s. Scott Marlowe ha scritto: > On Tue, Dec 9, 2008 at 5:17 AM, Mario Weilguni <mweilguni@sime.com> wrote: >> Alan Hodgson schrieb: >>>>>>> Mario Weilguni <mweilguni@sime.com> wrote: >>>>>>> >>>>> strange values. An individual drive is capable of delivering 91 >>>>> MB/sec >>>>> sequential read performance, and we get values ~102MB/sec out of a >>>>> 8-drive RAID5, seems to be ridiculous slow. >>> >>> What command are you using to test the reads? >>> >>> Some recommendations to try: >>> >>> 1) /sbin/blockdev --setra 2048 device (where device is the partition or >>> LVM volume) >>> >>> 2) Use XFS, and make sure your stripe settings match the RAID. >>> >>> Having said that, 102MB/sec sounds really low for any modern controller >>> with 8 drives, regardless of tuning or filesystem choice. >>> >>> >> First, thanks alot for this and all the other answers. >> >> I measured the raw device performance: >> dd if=/dev/cciss/c0d0 bs=64k count=100000 of=/dev/null >> >> I get poor performance when all 8 drives are configured as one, large >> RAID-5, and slightly poorer performance when configured as JBOD. In >> production, we use XFS as FS, but I doubt this has anything to do with FS >> tuning. > > Yeah, having just trawled the pgsql-performance archives, there are > plenty of instances of people having terrible performance from HP > smart array controllers before the P800. Is it possible for you to > trade up to a better RAID controller? Whichever salesman sold you the > P400 should take one for the team and make this right for you. >
Attachment
* Joshua D. Drake <jd@commandprompt.com> [081209 11:01]: > Yes the SmartArray series is quite common and actually know to perform > reasonably well, in RAID 10. You still appear to be trying RAID 5. *boggle* Are people *still* using raid5? /me gives up! -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Attachment
On Tue, Dec 9, 2008 at 9:03 AM, Gabriele Turchi <gabriele.turchi@l39a.com> wrote: > We reached a fairly good performance on a P400 controller (8 SATA 146GB 2,5" > 10k rpm) with raid5 or raid6 Linux software raid: the writing bandwidth > reached about 140 MB/s sustained throughput (the hardware raid5 gave a > sustained 20 MB/s...). With a second, equal controller (16 disks) we reached > (raid6 spanning all 16 disks) about 200 MB/s sustained. That's better than you were getting but still quite slow. I was bothered that my 12x15k4 SAS RAID-10 array could only sustain about 350Megs/second sequential read, thinking that each drive should be able to approach 80 or so megs/second and I was only getting about 60... This sounds more and more like HP is trying to undercompete along with Dell in the RAID controller market or at least the raid controller driver market.
Aidan Van Dyk wrote: > * Joshua D. Drake <jd@commandprompt.com> [081209 11:01]: > >> Yes the SmartArray series is quite common and actually know to perform >> reasonably well, in RAID 10. You still appear to be trying RAID 5. > > *boggle* > > Are people *still* using raid5? > > /me gives up! What do you suggest when there is not enough room for a RAID 10?
* Peter Eisentraut <peter_e@gmx.net> [081209 11:28]: > What do you suggest when there is not enough room for a RAID 10? More disks ;-) But if you've given up on performance and reliability in favour of cheaper storage, I guess raid5 is ok. But then I'm not sure what the point of asking about it's poor performance is... a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Attachment
On Tue, 2008-12-09 at 18:27 +0200, Peter Eisentraut wrote: > Aidan Van Dyk wrote: > > * Joshua D. Drake <jd@commandprompt.com> [081209 11:01]: > > > >> Yes the SmartArray series is quite common and actually know to perform > >> reasonably well, in RAID 10. You still appear to be trying RAID 5. > > > > *boggle* > > > > Are people *still* using raid5? > > > > /me gives up! > > What do you suggest when there is not enough room for a RAID 10? RAID 1. Joshua D. Drake > -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Tue, 2008-12-09 at 09:25 -0700, Scott Marlowe wrote: > On Tue, Dec 9, 2008 at 9:03 AM, Gabriele Turchi > <gabriele.turchi@l39a.com> wrote: > > We reached a fairly good performance on a P400 controller (8 SATA 146GB 2,5" > > 10k rpm) with raid5 or raid6 Linux software raid: the writing bandwidth > > reached about 140 MB/s sustained throughput (the hardware raid5 gave a > > sustained 20 MB/s...). With a second, equal controller (16 disks) we reached > > (raid6 spanning all 16 disks) about 200 MB/s sustained. > > That's better than you were getting but still quite slow. I was > bothered that my 12x15k4 SAS RAID-10 array could only sustain about > 350Megs/second sequential read, thinking that each drive should be > able to approach 80 or so megs/second and I was only getting about > 60... > > This sounds more and more like HP is trying to undercompete along with > Dell in the RAID controller market or at least the raid controller > driver market. It is certainly possible. The 400 is the higher end of the lower end with HP... 200, 400, 600, 800 (800 is a nice controller). Joshua D. Drake > -- PostgreSQL Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
Scott Marlowe schrieb: > On Tue, Dec 9, 2008 at 5:17 AM, Mario Weilguni <mweilguni@sime.com> wrote: > >> Alan Hodgson schrieb: >> >>>>>>> Mario Weilguni <mweilguni@sime.com> wrote: >>>>>>> >>>>>>> >>>>> strange values. An individual drive is capable of delivering 91 >>>>> MB/sec >>>>> sequential read performance, and we get values ~102MB/sec out of a >>>>> 8-drive RAID5, seems to be ridiculous slow. >>>>> >>> What command are you using to test the reads? >>> >>> Some recommendations to try: >>> >>> 1) /sbin/blockdev --setra 2048 device (where device is the partition or >>> LVM volume) >>> >>> 2) Use XFS, and make sure your stripe settings match the RAID. >>> >>> Having said that, 102MB/sec sounds really low for any modern controller >>> with 8 drives, regardless of tuning or filesystem choice. >>> >>> >>> >> First, thanks alot for this and all the other answers. >> >> I measured the raw device performance: >> dd if=/dev/cciss/c0d0 bs=64k count=100000 of=/dev/null >> >> I get poor performance when all 8 drives are configured as one, large >> RAID-5, and slightly poorer performance when configured as JBOD. In >> production, we use XFS as FS, but I doubt this has anything to do with FS >> tuning. >> > > Yeah, having just trawled the pgsql-performance archives, there are > plenty of instances of people having terrible performance from HP > smart array controllers before the P800. Is it possible for you to > trade up to a better RAID controller? Whichever salesman sold you the > P400 should take one for the team and make this right for you. > > A customer of us uses the P400 on a different machine, 8 SAS drives (Raid 5 as well), and the performance is very, very good. So we thought it's a good choice. Maybe the SATA drives are the root of this problem?
It could be the drives, it could be a particular interaction between them and the drivers or firmware.
Do you know if NCQ is activated for them?
Can you test a single drive JBOD through the array to the same drive through something else, perhaps the motherboard’s SATA port?
You may also have better luck with software raid-0 on top of 2 4 disk raid 5’s or raid 10s. But not if a single disk JBOD still performs well under par. You may need new drivers for the card, or firmware for the drive and or card. Or, the card may simply be incompatible with those drives. I’ve seen several hard drive – raid card incompatibilities before.
On 12/9/08 11:45 PM, "Mario Weilguni" <mweilguni@sime.com> wrote:
Do you know if NCQ is activated for them?
Can you test a single drive JBOD through the array to the same drive through something else, perhaps the motherboard’s SATA port?
You may also have better luck with software raid-0 on top of 2 4 disk raid 5’s or raid 10s. But not if a single disk JBOD still performs well under par. You may need new drivers for the card, or firmware for the drive and or card. Or, the card may simply be incompatible with those drives. I’ve seen several hard drive – raid card incompatibilities before.
On 12/9/08 11:45 PM, "Mario Weilguni" <mweilguni@sime.com> wrote:
Scott Marlowe schrieb:
> On Tue, Dec 9, 2008 at 5:17 AM, Mario Weilguni <mweilguni@sime.com> wrote:
>
>> Alan Hodgson schrieb:
>>
>>>>>>> Mario Weilguni <mweilguni@sime.com> wrote:
>>>>>>>
>>>>>>>
>>>>> strange values. An individual drive is capable of delivering 91
>>>>> MB/sec
>>>>> sequential read performance, and we get values ~102MB/sec out of a
>>>>> 8-drive RAID5, seems to be ridiculous slow.
>>>>>
>>> What command are you using to test the reads?
>>>
>>> Some recommendations to try:
>>>
>>> 1) /sbin/blockdev --setra 2048 device (where device is the partition or
>>> LVM volume)
>>>
>>> 2) Use XFS, and make sure your stripe settings match the RAID.
>>>
>>> Having said that, 102MB/sec sounds really low for any modern controller
>>> with 8 drives, regardless of tuning or filesystem choice.
>>>
>>>
>>>
>> First, thanks alot for this and all the other answers.
>>
>> I measured the raw device performance:
>> dd if=/dev/cciss/c0d0 bs=64k count=100000 of=/dev/null
>>
>> I get poor performance when all 8 drives are configured as one, large
>> RAID-5, and slightly poorer performance when configured as JBOD. In
>> production, we use XFS as FS, but I doubt this has anything to do with FS
>> tuning.
>>
>
> Yeah, having just trawled the pgsql-performance archives, there are
> plenty of instances of people having terrible performance from HP
> smart array controllers before the P800. Is it possible for you to
> trade up to a better RAID controller? Whichever salesman sold you the
> P400 should take one for the team and make this right for you.
>
>
A customer of us uses the P400 on a different machine, 8 SAS drives
(Raid 5 as well), and the performance is very, very good. So we thought
it's a good choice. Maybe the SATA drives are the root of this problem?
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Aidan Van Dyk schrieb: > * Joshua D. Drake <jd@commandprompt.com> [081209 11:01]: > > >> Yes the SmartArray series is quite common and actually know to perform >> reasonably well, in RAID 10. You still appear to be trying RAID 5. >> > > *boggle* > > Are people *still* using raid5? > > /me gives up! > > Why not? I know it's not performing as good as RAID-10, but it does not waste 50% diskspace. RAID-6 is no option, because the performance is even worse. And, on another system with RAID-5 + spare and SAS drives, the same controller is working very well.
On Wed, Dec 10, 2008 at 5:29 AM, Mario Weilguni <mweilguni@sime.com> wrote: > Aidan Van Dyk schrieb: >> >> * Joshua D. Drake <jd@commandprompt.com> [081209 11:01]: >> >>> >>> Yes the SmartArray series is quite common and actually know to perform >>> reasonably well, in RAID 10. You still appear to be trying RAID 5. >>> >> >> *boggle* >> Are people *still* using raid5? >> >> /me gives up! >> >> > > Why not? I know it's not performing as good as RAID-10, but it does not > waste 50% diskspace. RAID-6 is no option, because the performance is even > worse. And, on another system with RAID-5 + spare and SAS drives, the same > controller is working very well. I wouldn't refer to it as "waste". It's a tradeoff. With RAID-10 you get good performance at a cost of having 1/2 the storage capacity of your drives combined. RAID-5 says you're more worried about your budget than performance, and sometimes that's the case. The production servers where I work run a 25G database on a 12 disk RAID-10. The fact that we get less than a terabyte from 12 147G scsi disks is no great loss to us, we're interested in having a disk array that can handle a few 100 transactions per second even if a disk dies. Also, RAID-6 is faster, IF one of your disks has died (and you're on a good RAID controller). RAID-5 degraded performance is abysmal even on good controllers. RAID-10 with a lost disk is about the same as RAID-10 with all its disks in terms of performance. If you're running a production 24/7 database server, you can't afford to lose 80%+ of your throughput when a single drive fails. But this has all been covered before (many times) both on this list and over the internet. RAID-5 is useful for large data stores that can afford downtime. And there are plenty of apps like that. If your app isn't like that, i.e. needs to be up and performing well 24/7, or close to it, then RAID5 is a tragic mistake to make.
On Wed, Dec 10, 2008 at 12:45 AM, Mario Weilguni <mweilguni@sime.com> wrote: > > A customer of us uses the P400 on a different machine, 8 SAS drives (Raid 5 > as well), and the performance is very, very good. So we thought it's a good > choice. Maybe the SATA drives are the root of this problem? What tests have you or the customer done to confirm that performance is very very good? A lot of times the system is not as fast as the customer thinks, it's just faster than it was before and they're happy. Also, there could be problems in the driver or firmware on your P400 versus the customer one. I'd look for those differences as well. I doubt SATA versus SAS is the problem, but who knows...
Scott Marlowe schrieb: > On Wed, Dec 10, 2008 at 12:45 AM, Mario Weilguni <mweilguni@sime.com> wrote: > >> A customer of us uses the P400 on a different machine, 8 SAS drives (Raid 5 >> as well), and the performance is very, very good. So we thought it's a good >> choice. Maybe the SATA drives are the root of this problem? >> > > What tests have you or the customer done to confirm that performance > is very very good? A lot of times the system is not as fast as the > customer thinks, it's just faster than it was before and they're > happy. Also, there could be problems in the driver or firmware on > your P400 versus the customer one. I'd look for those differences as > well. I doubt SATA versus SAS is the problem, but who knows... > > Well, I cannot take the box offline to make usefull tests like tiobench or bonnie, but even with the current service running I get from a simple dd between 270 and 340 MB/sec sustained read over 30% of the disk. It also performed extremly good when I put the box into production, pg_bench values were impressing, but I do not have them at hand. However, currently we are seriously considering dropping RAID5 in favor of RAID10, we will test this week if this performs better. Regards Mario
Me, I'd *never) allow RAID-5 for data I cared abut.
Among other references, see <http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt>
(sorry for top-posting -- challenged reader and no other content to add)
Greg Williamson
Senior DBA
DigitalGlobe
Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information and must be protected in accordance with those provisions. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
(My corporate masters made me say this.)
-----Original Message-----
From: pgsql-performance-owner@postgresql.org on behalf of Mario Weilguni
Sent: Wed 12/10/2008 4:29 AM
To: Aidan Van Dyk
Cc: Joshua D. Drake; Scott Marlowe; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Experience with HP Smart Array P400 and SATA drives?
Aidan Van Dyk schrieb:
> * Joshua D. Drake <jd@commandprompt.com> [081209 11:01]:
>
>
>> Yes the SmartArray series is quite common and actually know to perform
>> reasonably well, in RAID 10. You still appear to be trying RAID 5.
>>
>
> *boggle*
>
> Are people *still* using raid5?
>
> /me gives up!
>
>
Why not? I know it's not performing as good as RAID-10, but it does not
waste 50% diskspace. RAID-6 is no option, because the performance is
even worse. And, on another system with RAID-5 + spare and SAS drives,
the same controller is working very well.
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
* Mario Weilguni <mweilguni@sime.com> [081210 07:31]: > Why not? I know it's not performing as good as RAID-10, but it does not > waste 50% diskspace. RAID-6 is no option, because the performance is > even worse. And, on another system with RAID-5 + spare and SAS drives, > the same controller is working very well. Like Scott said, it's all about trade-offs. With raid5, you get abysmal write performance, "make me not sleep at night" inconsistent parity issues, and a degraded mode that will a nightmare ... ... and as a trade-off you save a little money, and get good "read only" performance ... ... as long as you don't ever have a disk or system crash ... ... or can afford to rebuild if you do ... ... etc ... -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Attachment
Aidan Van Dyk schrieb: > * Mario Weilguni <mweilguni@sime.com> [081210 07:31]: > > >> Why not? I know it's not performing as good as RAID-10, but it does not >> waste 50% diskspace. RAID-6 is no option, because the performance is >> even worse. And, on another system with RAID-5 + spare and SAS drives, >> the same controller is working very well. >> > > Like Scott said, it's all about trade-offs. > > With raid5, you get abysmal write performance, "make me not sleep at > night" inconsistent parity issues, and a degraded mode that will a > nightmare ... > > ... and as a trade-off you save a little money, and get good "read only" > performance ... > > ... as long as you don't ever have a disk or system crash ... > > ... or can afford to rebuild if you do ... > > ... etc ... > In fact, for this system we're currently going to RAID10, I'm convinced now. With other systems we have, RAID5 is a safe option for one reason, the machines are clusters, so we have (sort of) RAID50 here: Machine A/RAID5 <-- DRBD --> Machine B/RAID5 Seems reliable enough for me. But in this case, the machine will be standalone, and so RAID5 might really not be the best choice. However, I'm pretty sure we'll have the same problems with RAID10, the problem seems to have to do with P400 and/or SATA drives.
On Wed, Dec 10, 2008 at 9:05 AM, Mario Weilguni <mweilguni@sime.com> wrote: > In fact, for this system we're currently going to RAID10, I'm convinced now. > With other systems we have, RAID5 is a safe option for one reason, the > machines are clusters, so we have (sort of) RAID50 here: > Machine A/RAID5 <-- DRBD --> Machine B/RAID5 > > Seems reliable enough for me. But in this case, the machine will be > standalone, and so RAID5 might really not be the best choice. > > However, I'm pretty sure we'll have the same problems with RAID10, the > problem seems to have to do with P400 and/or SATA drives. Yeah, I'm thinking there's something off in your system and until you resolve it you're going to have issues. I'd check the following: firmware on RAID controller how it runs with a couple of SAS drives in RAID-1 or RAID-0 (just for testing) OS version / kernel version / driver version. especially compared to your customer's machine. See how much of his environment you can clone until performance goes up where it should be. Then change one thing at a time until you break it again. I'm sure everyone here would like to know what makes a P400 fast or slow. Or, if you don't have time to mess with it, just order an escalade or areca card and be done with it. :)
On Wed, 10 Dec 2008, Scott Marlowe wrote:
> Or, if you don't have time to mess with it, just order an escalade or
> areca card and be done with it. :)
I'd be interested in recommendations for RAID cards for small SATA
systems. It's not anything to do with Postgres - I'm just intending to set
up a little four-drive array for my home computer, with cheap 1TB SATA
drives.
What PCI-Express or motherboard built-in SATA RAID controllers for about
four drives are there out there that are good, and well supported by
Linux? What level of support is there for monitoring and reporting of RAID
status?
Also, is it possible to set the drives in a hardware RAID into
auto-spindown mode? I'm going to be putting a SSD in as the main system
drive, with the RAID array to hold my large stuff which I only work on
part of the time.
Matthew
--
Failure is not an option. It comes bundled with your Microsoft product.
-- Ferenc Mantfeld
On Wed, 10 Dec 2008, Matthew Wakeling wrote: > I'd be interested in recommendations for RAID cards for small SATA systems. > It's not anything to do with Postgres - I'm just intending to set up a little > four-drive array for my home computer, with cheap 1TB SATA drives. Then why are you thinking of RAID cards? On a Linux only host, you might as well just get a standard cheap multi-port SATA card that's compatible with the OS, plug the four drives in, and run software RAID. Anything else you put in the middle is going to add complications in terms of things like getting SMART error data from the drives, and the SW RAID will probably be faster too. A great source for checking Linux compatibility is http://linux-ata.org/driver-status.html The only reason I have a good controller card in my system at home is because it lets me do realistic performance tests with a write cache, so fsync is accelerated when working with PostgreSQL. If just putting a bunch of drives in there was my only concern, I'd have just bought a cheap Silicon Image 3124 PCI-Express board. > What PCI-Express or motherboard built-in SATA RAID controllers for about four > drives are there out there that are good, and well supported by Linux? What > level of support is there for monitoring and reporting of RAID status? 3ware 9650SE-4LPML is what I'd buy today if I wanted hardware SATA RAID. When I made a similar decision some time ago, I bought an Areca 1210 instead, but two things have changed since then. One, I've become increasingly unsatisfied with the limitations of the closed-source controller management tool Areca supplies. And the performance of 3ware's earlier 9550 model really lagged relative to Areca, while the newer 9650 is quite competative. More details on all of that on the blog entry I wrote after my last disk failure: http://notemagnet.blogspot.com/2008/08/linux-disk-failures-areca-is-not-so.html -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Wed, Dec 10, 2008 at 12:13 PM, Greg Smith <gsmith@gregsmith.com> wrote: > On Wed, 10 Dec 2008, Matthew Wakeling wrote: > >> I'd be interested in recommendations for RAID cards for small SATA >> systems. It's not anything to do with Postgres - I'm just intending to set >> up a little four-drive array for my home computer, with cheap 1TB SATA >> drives. > > Then why are you thinking of RAID cards? On a Linux only host, you might as > well just get a standard cheap multi-port SATA card that's compatible with > the OS, plug the four drives in, and run software RAID. Anything else you > put in the middle is going to add complications in terms of things like > getting SMART error data from the drives, and the SW RAID will probably be > faster too. Note that you could combine the two and use a caching controller in jbod mode and do the raid in linux kernel sw mode. Just a thought. Not sure about the smart stuff though. >> What PCI-Express or motherboard built-in SATA RAID controllers for about >> four drives are there out there that are good, and well supported by Linux? >> What level of support is there for monitoring and reporting of RAID status? > > 3ware 9650SE-4LPML is what I'd buy today if I wanted hardware SATA RAID. > When I made a similar decision some time ago, I bought an Areca 1210 > instead, but two things have changed since then. One, I've become > increasingly unsatisfied with the limitations of the closed-source > controller management tool Areca supplies. Note that Areca's newest controllers, the 1680 series, have a separate ethernet port with snmp traps so you don't have to use any special closed source software to monitor them. Just FYI for anyone considering them.
On Wed, Dec 10, 2008 at 2:13 PM, Greg Smith <gsmith@gregsmith.com> wrote: > 3ware 9650SE-4LPML is what I'd buy today if I wanted hardware SATA RAID. FWIW, I just put together a system with exactly that (4 320g drives in raid 10) and have been pleased with the results. I won't have any downtime to be able to get performance benchmarks until the current compute/write pass finishes in a week and a half or so, but I don't have any complaints with how it's performing for our app. DB size is ~120gb now and add ~7gb/day during the current phase, at which point it'll move to a light-write, high-read data warehouse style usage pattern. -- - David T. Wilson david.t.wilson@gmail.com
On Wed, 10 Dec 2008, Greg Smith wrote: >> I'd be interested in recommendations for RAID cards for small SATA systems. >> It's not anything to do with Postgres - I'm just intending to set up a >> little four-drive array for my home computer, with cheap 1TB SATA drives. > > Then why are you thinking of RAID cards? On a Linux only host, you might as > well just get a standard cheap multi-port SATA card that's compatible with > the OS, plug the four drives in, and run software RAID. Anything else you > put in the middle is going to add complications in terms of things like > getting SMART error data from the drives, and the SW RAID will probably be > faster too. > > A great source for checking Linux compatibility is > http://linux-ata.org/driver-status.html Thanks, that is the kind of info I was looking for. It looks like most sensible SATA controller manufacturers are converging towards the open ahci controller standard, which is useful. Matthew -- The best way to accelerate a Microsoft product is at 9.8 metres per second per second. - Anonymous