Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins - Mailing list pgsql-performance

From Scott Carey
Subject Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
Date
Msg-id 2CE778F3-1147-4AC3-AA33-1911EDDD7AB9@richrelevance.com
Whole thread Raw
In response to Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins  (Digimer <linux@alteeve.com>)
Responses Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
List pgsql-performance
On Nov 17, 2010, at 7:28 AM, Digimer wrote:

> On 11/17/2010 09:26 AM, Eric Comeau wrote:
>> This is not directly a PostgreSQL performance question but I'm hoping
>> some of the chaps that build high IO PostgreSQL servers on here can help.
>>
>> We build file transfer acceleration s/w (and use PostgreSQL as our
>> database) but we need to build a test server that can handle a sustained
>> write throughput of 1,25 GB for 5 mins.
>>
>> Why this number, because we want to push a 10 Gbps network link for 5-8
>> mins, 10Gbps = 1.25 GB write, and would like to drive it for 5-8 mins
>> which would be 400-500 GB.
>>
>> Note this is just a "test" server therefore it does not need fault
>> tolerance.
>>
>> Thanks in advance,
>> Eric
>>
>
> Off hand, I would suggest:
>
> 8x http://www.kingston.com/ssd/vplus100.asp (180MB/sec sustained write)
> stripped (RAID 0, you did say that you don't care about safety). That
> should be 1.44GB/sec write, minus overhead.

Can get cheaper disks that go ~135MB/sec write and a couple more of them.

>
> 1x
> http://www.lsi.com/channel/products/raid_controllers/3ware_9690sa8i/index.html
> RAID card (note that it's the internal port model, despite the image)
>

You'll need 2 RAID cards with software raid-0 on top to sustain this rate, or simply pure software raid-0.  A single
raidcard tends to be unable to sustain reads or writes that high, no matter how many drives you put on it. 

The last time I tried a 3ware card, it couldn't go past 380MB/sec with 10 drives. 6 to 10 drives in raid 10 were all
thesame sequential througput, only random iops went up.  Maybe raid0 is better.  Software raid is usually fastest for
raid0, 1, and 10, other than write cache effects (which are strong and important for a real world db). 

I get ~1000MB/sec out of 2 Adaptec 5805s with linux 'md' software raid 0 on top of these (each are raid 10 with 10
drives). If i did not care about data reliability I'd go with anything that had a lot of ports (perhaps a couple cheap
SAScards without complicated raid features) and software raid 0. 


> 4x http://usa.chenbro.com/corporatesite/products_detail.php?sku=114 (for
> mounting the drives)
>
> That would be about the minimum I should expect you can pay to get that
> kind of performance. Others are free to dis/agree. :)
>
> --
> Digimer
> E-Mail: digimer@alteeve.com
> AN!Whitepapers: http://alteeve.com
> Node Assassin:  http://nodeassassin.org
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance


pgsql-performance by date:

Previous
From: Josh Berkus
Date:
Subject: Anyone seen this kind of lock pileup?
Next
From: Scott Carey
Date:
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins