Re: Optimal settings for RAID controller - optimized for writes

From: Tomas Vondra
Subject: Re: Optimal settings for RAID controller - optimized for writes
Date: ,
Msg-id: 530555F5.7070604@fuzzy.cz
(view: Whole thread, Raw)
In response to: Re: Optimal settings for RAID controller - optimized for writes  (Scott Marlowe)
Responses: Re: Optimal settings for RAID controller - optimized for writes  (Scott Marlowe)
List: pgsql-performance

Tree view

Optimal settings for RAID controller - optimized for writes  (Niels Kristian Schjødt, )
 Re: Optimal settings for RAID controller - optimized for writes  (DFE, )
  Re: Optimal settings for RAID controller - optimized for writes  ("Tomas Vondra", )
   Re: Optimal settings for RAID controller - optimized for writes  (KONDO Mitsumasa, )
    Re: Optimal settings for RAID controller - optimized for writes  (Tomas Vondra, )
     Re: Optimal settings for RAID controller - optimized for writes  (KONDO Mitsumasa, )
      Re: Optimal settings for RAID controller - optimized for writes  (Tomas Vondra, )
       Re: Optimal settings for RAID controller - optimized for writes  (KONDO Mitsumasa, )
     Re: Optimal settings for RAID controller - optimized for writes  (Merlin Moncure, )
      Re: Optimal settings for RAID controller - optimized for writes  (Scott Marlowe, )
       Re: Optimal settings for RAID controller - optimized for writes  (Merlin Moncure, )
       Re: Optimal settings for RAID controller - optimized for writes  (Tomas Vondra, )
        Re: Optimal settings for RAID controller - optimized for writes  (Scott Marlowe, )
         Re: Optimal settings for RAID controller - optimized for writes  (Tomas Vondra, )
      Re: Optimal settings for RAID controller - optimized for writes  (Tomas Vondra, )
  Re: Optimal settings for RAID controller - optimized for writes  (Michael Stone, )
 Re: Optimal settings for RAID controller - optimized for writes  ("Tomas Vondra", )
 Re: Optimal settings for RAID controller - optimized for writes  (Scott Marlowe, )

On 19.2.2014 19:09, Scott Marlowe wrote:
>
> You can also do the old trick of underprovisioning and / or
> underutilizing all the space on SSDs. I.e. put 10 600GB SSDs under a
> HW RAID controller in RAID-10, then only parititon out 1/2 the storage
> you get from that. so you get 1.5TB os storage and the drives are
> underutilized enough to have spare space.

Yeah. AFAIK that's basically what Intel did with S3500 -> S3700.

What I'm trying to find is the 'sweet spot' considering lifespan,
capacity, performance and price. That's why I'm still wondering if there
are some experiences with current generation of SSDs and RAID
controllers, with RAID levels other than RAID-10.

Say I have 8x 400GB SSD, 75k/32k read/write IOPS each (i.e. it's
basically the S3700 from Intel). Assuming the writes are ~25% of the
workload, this is what I get for RAID10 vs. RAID6 (math done using
http://www.wmarow.com/strcalc/)

            | capacity GB | bandwidth MB/s | IOPS
 ---------------------------------------------------
   RAID-10  |        1490 |           2370 | 300k
   RAID-6   |        2230 |           1070 | 130k

Let's say the app can't really generate 130k IOPS (we'll saturate CPU
way before that), so even if the real-world numbers will be less than
50% of this, we're not going to hit disks as the main bottleneck.

So let's assume there's no observable performance difference between
RAID10 and RAID6 in our case. But we could put 1.5x the amount of data
on the RAID6, making it much cheaper (we're talking about non-trivial
numbers of such machines).

The question is - how long will it last before the SSDs die because of
wearout? Will the RAID controller make it worse due to (not) handling
TRIM? Will we know how much time we have left, i.e. will the controller
provide the info the drives provide through SMART?

> Right now I'm testing on a machine with 2x Intel E5-2690s
> (http://ark.intel.com/products/64596/intel-xeon-processor-e5-2690-20m-cache-2_90-ghz-8_00-gts-intel-qpi)
> 512GB RAM and 6x600GB Intel SSDs (not sure which ones) under a LSI

Most likely S3500. S3700 are not offered with 600GB capacity.

> MegaRAID 9266. I'm able to crank out 6500 to 7200 TPS under pgbench on
> a scale 1000 db at 8 to 60 clients on that machine. It's not cheap,
> but storage wise it's WAY cheaper than most SANS and very fast.
> pg_xlog is on a pair of non-descript SATA spinners btw.

Nice. I've done some testing with fusionio iodrive duo (2 devices in
RAID0) ~ year ago, and I got 12k TPS (or ~15k with WAL on SAS RAID). So
considering the price, the 7.2k TPS is really good IMHO.

regards
Tomas



pgsql-performance by date:

From: Niels Kristian Schjødt
Date:
Subject: Optimal settings for RAID controller - optimized for writes
From: KONDO Mitsumasa
Date:
Subject: Re: Optimal settings for RAID controller - optimized for writes