Thread: FUSION-IO io cards
Hi,
Had a recent conversation with a tech from this company called FUSION-IO. They sell
io cards designed to replace conventional disks. The cards can be up to 3 TB in size and apparently
are installed in closer proximity to the CPU than the disks are. They claim
performance boosts several times better than the spinning disks.
Just wondering if anyone has had any experience with this company and these
cards. We're currently at postgres 8.3.11.
Any insights / recommendations appreciated. thank you,
--
Had a recent conversation with a tech from this company called FUSION-IO. They sell
io cards designed to replace conventional disks. The cards can be up to 3 TB in size and apparently
are installed in closer proximity to the CPU than the disks are. They claim
performance boosts several times better than the spinning disks.
Just wondering if anyone has had any experience with this company and these
cards. We're currently at postgres 8.3.11.
Any insights / recommendations appreciated. thank you,
--
Mark Steben
Database Administrator
@utoRevenue | Autobase | AVV
The CRM division of Dominion Dealer Solutions
95D Ashley Avenue
West Springfield, MA 01089
t: 413.327.3045
f: 413.732.1824
w: www.autorevenue.com
On 4/29/2011 10:24 AM, Mark Steben wrote: > Hi, > Had a recent conversation with a tech from this company called > FUSION-IO. They sell > io cards designed to replace conventional disks. The cards can be up > to 3 TB in size and apparently > are installed in closer proximity to the CPU than the disks are. They claim > performance boosts several times better than the spinning disks. > > Just wondering if anyone has had any experience with this company and these > cards. We're currently at postgres 8.3.11. > > Any insights / recommendations appreciated. thank you, Well, The Woz works there. Not because he needs money, but because he thinks they are doing it right.
On 29/04/2011 16:24, Mark Steben wrote: > Hi, > Had a recent conversation with a tech from this company called FUSION-IO. > They sell > io cards designed to replace conventional disks. The cards can be up to 3 > TB in size and apparently > are installed in closer proximity to the CPU than the disks are. They claim > performance boosts several times better than the spinning disks. > > Just wondering if anyone has had any experience with this company and these > cards. We're currently at postgres 8.3.11. > > Any insights / recommendations appreciated. thank you, They are actually very fast SSDs; the fact that they come in "card" format and not in conventional "box with plugs" format is better since they have less communication overhead and electrical interference. As far as I've heard, the hardware is as good as they say it is.
Fusion SSDs install on PCIe slots, so are limited by slot count. None, so far as I recall, are bootable (although Fusionhas been promising that for more than a year). If you've a BCNF schema of moderate size, then any SSD as primary storeis a good option; Fusion's are just even faster. If you've got the typical flatfile bloated schema, then while SSDwill be faster (if you've got the $$$), PCIe is not likely to have sufficient capacity. SSD is the reason to refactor to a Dr. Coddian schema. Few have taken the opportunity. regards, Robert ---- Original message ---- >Date: Fri, 29 Apr 2011 10:24:48 -0400 >From: pgsql-performance-owner@postgresql.org (on behalf of Mark Steben <mark.steben@autorevenue.com>) >Subject: [PERFORM] FUSION-IO io cards >To: pgsql-performance@postgresql.org > > Hi, > Had a recent conversation with a tech from this > company called FUSION-IO. They sell > io cards designed to replace conventional disks. > The cards can be up to 3 TB in size and apparently > are installed in closer proximity to the CPU than > the disks are. They claim > performance boosts several times better than the > spinning disks. > > Just wondering if anyone has had any experience with > this company and these > cards. We're currently at postgres 8.3.11. > > Any insights / recommendations appreciated. thank > you, > > -- > > Mark Steben > Database Administrator > @utoRevenue | Autobase | AVV > The CRM division of Dominion Dealer Solutions > 95D Ashley Avenue > West Springfield, MA 01089 > t: 413.327.3045 > f: 413.732.1824 > w: www.autorevenue.com
On Apr 29, 2011, at 7:24 AM, Mark Steben wrote:
Hi,
Had a recent conversation with a tech from this company called FUSION-IO. They sell
io cards designed to replace conventional disks. The cards can be up to 3 TB in size and apparently
are installed in closer proximity to the CPU than the disks are. They claim
performance boosts several times better than the spinning disks.
Just wondering if anyone has had any experience with this company and these
cards. We're currently at postgres 8.3.11.
Any insights / recommendations appreciated. thank you,
We have a bunch of their cards, purchased when we were still on 8.1 and were having difficulty with vacuums. (Duh.) They helped out a bunch for that. They're fast, no question about it. Each FusionIO device (they have cards with multiple devices) can do ~100k iops. So that's nifty.
On the downside, they're also somewhat exotic, in that they need special kernel drivers, so they're not as easy as just buying a bunch of drives. More negatively, they're $$$. And even more negatively, their drivers are inefficient - expect to dedicate a CPU core to doing whatever they need done.
In the "still undecided" category I'm somewhat worried about their longevity. They say they overprovision the amount of flash so that burnout isn't a problem, and at least it's not like competitors we've seen, which throttle your writes so that you don't burn out as fast. Of course, the only way to tell how long they'll really last is to use them a long time. We're only about 2 years into them so come back to me in 3 years about this. :) Also, while I would say they seem reliable (they have a supercap and succeeded every power-pull test we did) we just recently we've had some issues which appear to be fio driver-related that effectively brought our server down. Fusion thinks its our kernel parameters, but we're unconvinced, given the length of time we've run with the same kernel settings. I'm not yet ready to say these cards are unreliable, but I'm no longer willing to say they're problem-free, either. I would say, if you're going to buy them, make sure you get a support contract. We didn't, and the support we've gotten so far has not been as responsive and I would have expected from such an expensive product.
Overall, I would recommend them. But just realize you're buying the race car of the storage world, which implies 1) you'll go fast, 2) you'll spend $$$, and 3) you'll have interesting problems most other people do not have.
We use FusionIO products for PGSQL. They work in most linux distributions and even have beta FreeBSD drivers for those of us who prefer that OS. They cost a lot, perform really well, and FusionIO has great support for those of us who prefer not to use Windows or OS X, something that many other vendors can't and don't usually care about.
Tyler Mills
Network Operations Tools Technician
Pavlov Media Inc.
NOC On Call: 217.841.5045NOC main line: 217.353.3059
From: pgsql-performance-owner@postgresql.org [pgsql-performance-owner@postgresql.org] on behalf of Ben Chobot [bench@silentmedia.com]
Sent: Friday, April 29, 2011 9:54 AM
To: Mark Steben
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] FUSION-IO io cards
Sent: Friday, April 29, 2011 9:54 AM
To: Mark Steben
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] FUSION-IO io cards
On Apr 29, 2011, at 7:24 AM, Mark Steben wrote:
Hi,
Had a recent conversation with a tech from this company called FUSION-IO. They sell
io cards designed to replace conventional disks. The cards can be up to 3 TB in size and apparently
are installed in closer proximity to the CPU than the disks are. They claim
performance boosts several times better than the spinning disks.
Just wondering if anyone has had any experience with this company and these
cards. We're currently at postgres 8.3.11.
Any insights / recommendations appreciated. thank you,
We have a bunch of their cards, purchased when we were still on 8.1 and were having difficulty with vacuums. (Duh.) They helped out a bunch for that. They're fast, no question about it. Each FusionIO device (they have cards with multiple devices) can do ~100k iops. So that's nifty.
On the downside, they're also somewhat exotic, in that they need special kernel drivers, so they're not as easy as just buying a bunch of drives. More negatively, they're $$$. And even more negatively, their drivers are inefficient - expect to dedicate a CPU core to doing whatever they need done.
In the "still undecided" category I'm somewhat worried about their longevity. They say they overprovision the amount of flash so that burnout isn't a problem, and at least it's not like competitors we've seen, which throttle your writes so that you don't burn out as fast. Of course, the only way to tell how long they'll really last is to use them a long time. We're only about 2 years into them so come back to me in 3 years about this. :) Also, while I would say they seem reliable (they have a supercap and succeeded every power-pull test we did) we just recently we've had some issues which appear to be fio driver-related that effectively brought our server down. Fusion thinks its our kernel parameters, but we're unconvinced, given the length of time we've run with the same kernel settings. I'm not yet ready to say these cards are unreliable, but I'm no longer willing to say they're problem-free, either. I would say, if you're going to buy them, make sure you get a support contract. We didn't, and the support we've gotten so far has not been as responsive and I would have expected from such an expensive product.
Overall, I would recommend them. But just realize you're buying the race car of the storage world, which implies 1) you'll go fast, 2) you'll spend $$$, and 3) you'll have interesting problems most other people do not have.
On 04/29/2011 04:54 PM, Ben Chobot wrote: > We have a bunch of their cards, purchased when we were still on 8.1 and > were having difficulty with vacuums. (Duh.) They helped out a bunch for > that. They're fast, no question about it. Each FusionIO device (they > have cards with multiple devices) can do ~100k iops. So that's nifty. > > On the downside, they're also somewhat exotic, in that they need special > kernel drivers, so they're not as easy as just buying a bunch of drives. > More negatively, they're $$$. And even more negatively, their drivers > are inefficient - expect to dedicate a CPU core to doing whatever they > need done. I would recommend to have a look a Texas Memory Systems for a comparison. FusionIO does a lot of work in software, as Ben noted correctly, while TMS (their stuff is called RAMSAN) is a more all-in-hardware device. Haven't used TMS myself, but talked to people who do know and their experience with both products is that TMS is problem-free and has a more deterministic performance. And I have in fact benchmarked FusionIO and observed non-deterministic performance, which means performance goes down siginificantly on occasion - probably because some software-based house-keeping needs to be done. -- Joachim Worringen Senior Performance Architect International Algorithmic Trading GmbH
TMS RAMSAN is a DRAM device. TMS built DRAM SSDs going back decades, but have recently gotten into flash SSDs as well. The DRAM parts are in an order of magnitude more expensive than others' flash SSDs, gig by gig. Also, about as fast as offcpu storage gets. regards, Robert ---- Original message ---- >Date: Fri, 29 Apr 2011 18:04:17 +0200 >From: pgsql-performance-owner@postgresql.org (on behalf of Joachim Worringen <joachim.worringen@iathh.de>) >Subject: Re: [PERFORM] FUSION-IO io cards >To: pgsql-performance@postgresql.org > >On 04/29/2011 04:54 PM, Ben Chobot wrote: >> We have a bunch of their cards, purchased when we were still on 8.1 and >> were having difficulty with vacuums. (Duh.) They helped out a bunch for >> that. They're fast, no question about it. Each FusionIO device (they >> have cards with multiple devices) can do ~100k iops. So that's nifty. >> >> On the downside, they're also somewhat exotic, in that they need special >> kernel drivers, so they're not as easy as just buying a bunch of drives. >> More negatively, they're $$$. And even more negatively, their drivers >> are inefficient - expect to dedicate a CPU core to doing whatever they >> need done. > >I would recommend to have a look a Texas Memory Systems for a >comparison. FusionIO does a lot of work in software, as Ben noted >correctly, while TMS (their stuff is called RAMSAN) is a more >all-in-hardware device. > >Haven't used TMS myself, but talked to people who do know and their >experience with both products is that TMS is problem-free and has a more >deterministic performance. And I have in fact benchmarked FusionIO and >observed non-deterministic performance, which means performance goes >down siginificantly on occasion - probably because some software-based >house-keeping needs to be done. > >-- >Joachim Worringen >Senior Performance Architect > >International Algorithmic Trading GmbH > > >-- >Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) >To make changes to your subscription: >http://www.postgresql.org/mailpref/pgsql-performance
Ben Chobot wrote: > Also, while I would say they seem reliable (they have a supercap and > succeeded every power-pull test we did) we just recently we've had > some issues which /appear/ to be fio driver-related that effectively > brought our server down. Fusion thinks its our kernel parameters, but > we're unconvinced, given the length of time we've run with the same > kernel settings. I'm not yet ready to say these cards are unreliable, > but I'm no longer willing to say they're problem-free, either. Ben has written a nice summary of the broader experience of everyone I've talked to who has deployed Fusion IO. The race car anology is a good one. Fast, minimal concerns about data loss, but occasional quirky things that are frustrating to track down and eliminate. Not much transparency in terms of what it's doing under the hood, which makes long-term reliability a concern too. A particularly regular complaint is that there are situations where the card can requite a long consistency check time on system boot after a crash. Nothing lost, but a long (many minutes) delay before the server is functioning again is possible. The already mentioned TI RAMSAN at an ever higher price point is also a possibility. Another more recent direct competitor to FusionIO's products comes from Virident: http://www.virident.com/ They seem to be doing everything right to make a FusionIO competitor at the same basic price point. They've already released good MySQL performance numbers, and they tell me that PostgreSQL ones are done but just not published yet; going through validation still. The "Performance Relative to Capacity Used" graph at http://www.ssdperformanceblog.com/2010/12/write-performance-on-virident-tachion-card/ is one anyone deploying on FusionIO should also be aware of. -- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
On 04/29/2011 06:52 PM, gnuoytr@rcn.com wrote: > TMS RAMSAN is a DRAM device. TMS built DRAM SSDs going back decades, > but have recently gotten into flash SSDs as well. The DRAM parts are > in an order of magnitude more expensive than others' flash SSDs, gig > by gig. Also, about as fast as off cpu storage gets. Their naming convention is a bit confusing, but in fact the RamSan boxes are available in flash and RAM-based variants: "The RamSan-630 offers 10 TB SLC Flash storage, 1,000,000 IOPS (10 GB/s) random sustained throughput, and just 500 watts power consumption." I was referring to those. Of course, they may be more expensive than FusionIO. You get what you pay for (in this case). -- Joachim Worringen Senior Performance Architect International Algorithmic Trading GmbH
On Fri, Apr 29, 2011 at 10:24 AM, Mark Steben <mark.steben@autorevenue.com> wrote: > Just wondering if anyone has had any experience with this company and these > cards. We're currently at postgres 8.3.11. td;dr Ask for a sample and test it out for yourself. I asked for, and received, a sample 80GB unit from Fusion to test out. Due to my own time constraints, I did not get to do nearly the testing I wanted to perform. Anecdotally, they are bloody fast. I ran an intense OLTP workload for 8.4 on it, and the card far exceeded anything I've seen elsewhere. I did see the CPU utilization that is mentioned elsewhere in this thread, and the drivers are exotic enough ( again repeating things mentioned elsewhere ) that I couldn't just load them up on any old linux distro I wanted to. The pre-sales engineer assigned to me - Sarit Birzon - was very helpful and nice.