Thread: Suggestions for Intel 710 SSD test

I have a 710 (Lyndonville) SSD in a test server. Ultimately we'll run
capacity tests using our application (which in turn uses PG), but it'll
take a while to get those set up. In the meantime, I'd be happy to
entertain running whatever tests folks here would like to suggest,
spare time-permitting.

I've already tried bonnie++, sysbench and a simple WAL emulation
test program I wrote more than 10 years ago.  The drive tests at
around 160Mbyte/s on bulk data and 4k tps for commit rate writing
small blocks.

Anandtech took the trouble of doing that:
http://www.anandtech.com/show/4902/intel-ssd-710-200gb-review

I think the main advantage of the 710 compared to the 320 is its much
heavier over-provisioning and better quality MLC-chips. Both the 320 and
710 use the same controller and offer similar performance. But 320GB of
raw capacity is sold as a 300GB Intel 320 and as a 200GB Intel 710...

So if you don't need write-endurance, you can probably assume the 320
will be more capacity and bang for the buck and will be good enough. If
you're a worried about write-endurance, you should have a look at the
710. You can obviously also only provision about 200GB of that 300GB
320-ssd and thus increase its expected live span, but you'd still miss
the higher quality MLC. Given the fact that you can get two 320's for
the price of one 710, its probably always a bit difficult to actually
make the choice (unless you want a fixed amount of disks and the best
endurance possible for that).

Best regards,

Arjen

On 2-10-2011 5:22 Andy wrote:
> Do you have an Intel 320? I'd love to see tests comparing 710 to 320 and
> see if it's worth the price premium.
>
> ------------------------------------------------------------------------
> *From:* David Boreham <david_list@boreham.org>
> *To:* PGSQL Performance <pgsql-performance@postgresql.org>
> *Sent:* Saturday, October 1, 2011 10:39 PM
> *Subject:* [PERFORM] Suggestions for Intel 710 SSD test
>
>
> I have a 710 (Lyndonville) SSD in a test server. Ultimately we'll run
> capacity tests using our application (which in turn uses PG), but it'll
> take a while to get those set up. In the meantime, I'd be happy to
> entertain running whatever tests folks here would like to suggest,
> spare time-permitting.
>
> I've already tried bonnie++, sysbench and a simple WAL emulation
> test program I wrote more than 10 years ago. The drive tests at
> around 160Mbyte/s on bulk data and 4k tps for commit rate writing
> small blocks.
>
>
>
> -- Sent via pgsql-performance mailing list
> (pgsql-performance@postgresql.org <mailto:pgsql-performance@postgresql.org>)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
>

On 10/1/2011 10:00 PM, Gregory Gerard wrote:
> How does this same benchmark compare on similar (or same) hardware but
> with magnetic media?

I don't have that data at present :(

So far I've been comparing performance with our current production
machines, which are older.
Those machines use 'raptors with no BBU/WBC, so their tps performance is
very low (100 per second or so).
Since our application tends to be limited by the transaction commit
rate, the SSD-based machine
is clearly going to be at least an order of magnitude faster. But I
don't know its relative performance
vs. the traditional BBU/WBC controller + string of drives approach (we
rejected that as an option already
for power consumption and space reasons).

I could install a raptor drive in the new server and test that, but it
wouldn't have a BBU/WBC controller.
I guess I could enable the drive's write cache which might produce
comparable results to a controller's WBC.

On 10/1/2011 9:22 PM, Andy wrote:
> Do you have an Intel 320?  I'd love to see tests comparing 710 to 320
> and see if it's worth the price premium.
>
Good question. I don't have a 320 drive today, but will probably get one
for testing soon.

However, my conclusion based on the Intel spec documents is that the 710
and 320 will have similar performance.
We elected to use 710 devices not for performance reasons vs the 320 but
for the specified lifetime endurance.
The 710 series is specified at around 4k complete drive overwrites where
the 320 is specified at only
around 100. So I see the 710 series as "like the 320 but much less
likely to wear out". It may also have
better behavior under constant load (the white paper makes vague mention
of different background GC
vs the non-enterprise drives).

So for our uses, the 320 series looks great except for concerns that :
a) the may wear out quite quickly, leading to extra cost to enter data
center and pull drives, etc and the need to maintain a long-term test
rig to determine if and when they wear out before it happens in production.
b) the GC may behave badly under constant load (leading for example to
unexpected periods of relatively very poor performance).

The value proposition for the 710 vs. 320 for me is not performance but
the avoidance of these two worries.

On 10/2/2011 2:33 AM, Arjen van der Meijden wrote:
>
>
> Given the fact that you can get two 320's for the price of one 710,
> its probably always a bit difficult to actually make the choice
> (unless you want a fixed amount of disks and the best endurance
> possible for that).

One thing I'd add to this is that the price/bit is more like 4X ($2k for
the 300G 710 vs $540 for the 300G 320).
The largest 710 drive is 300G whereas the largest 320 is 600G which may
imply that the 710's are twice
as over-provisioned as the 320. It may be that at present we're paying
2x for the relative over-provisioning
and another 2x to enjoy the better silicon and firmware. This hopefully
implies that prices will fall
in the future provided a credible competitor emerges (Hitachi??).

If I may ask what were your top three candidates before choosing the intel?

Also why not just plan a graceful switch to a replicated server? At some point you have to detect the drive is about to
go(or it just goes without warning). Presumably that point will be in a while and be coordinated with an upgrade like
9.2in a year. 

Finally why not the pci based cards?

On Oct 2, 2011, at 16:33, David Boreham <david_list@boreham.org> wrote:

> On 10/2/2011 2:33 AM, Arjen van der Meijden wrote:
>>
>>
>> Given the fact that you can get two 320's for the price of one 710, its probably always a bit difficult to actually
makethe choice (unless you want a fixed amount of disks and the best endurance possible for that). 
>
> One thing I'd add to this is that the price/bit is more like 4X ($2k for the 300G 710 vs $540 for the 300G 320).
> The largest 710 drive is 300G whereas the largest 320 is 600G which may imply that the 710's are twice
> as over-provisioned as the 320. It may be that at present we're paying 2x for the relative over-provisioning
> and another 2x to enjoy the better silicon and firmware. This hopefully implies that prices will fall
> in the future provided a credible competitor emerges (Hitachi??).
>
>
>
>
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

On 10/2/2011 6:26 PM, Gregory Gerard wrote:
> If I may ask what were your top three candidates before choosing the intel?
All the other options considered viable were using traditional
rotational disks.
I personally don't have any confidence in the other SSD vendors today,
except perhaps for FusionIO (where a couple of old friends work, and I
can certainly vouch for
their competence) but their products are too costly for our application
at present.
>
> Also why not just plan a graceful switch to a replicated server? At some point you have to detect the drive is about
togo (or it just goes without warning). Presumably that point will be in a while and be coordinated with an upgrade
like9.2 in a year. 
Sure, we have this capability but once you walk through what has to
happen if you are burning through
SSDs every few months, the 710 value proposition is more attractive for
us. For example our data center
is 1200 miles from our HQ and it takes a very long road trip or a plane
flight to get hands-on with the
boxes. We spent some considerable time planning for the 320 style
deployment to be honest --
figuring out how to predict when the drive would wear out, building
replication mechanisms that
would cope gracefully and so on. But given the option of the 710 where
wear out can essentially
be crossed off the list of things to worry about, that's the way we
decided to go.
>
> Finally why not the pci based cards?
>
Few reasons: 1) Today Intel doesn't make them (that will change soon),
2) a desire to maintain backwards compatibility at least for this
generation, on a system architecture level
with traditional disk drives, 3) concerns about mechanical integrity and
system airflow issues with the
PCI card and connector in 1U enclosures. The SSD fits into the same
location as a traditional disk
but can be velcro'ed down rather than bolted for easier field replacement.

On 10/01/2011 07:39 PM, David Boreham wrote:
> I've already tried bonnie++, sysbench and a simple WAL emulation
> test program I wrote more than 10 years ago.  The drive tests at
> around 160Mbyte/s on bulk data and 4k tps for commit rate writing
> small blocks.

That sounds about the same performance as the 320 drive I tested earlier
this year then.  You might try duplicating some of the benchmarks I ran
on that:
http://archives.postgresql.org/message-id/4D9D1FC3.4020207@2ndQuadrant.com

Make sure to reference the capacity of the drive though.  The 320 units
do scale their performance based on that, presumably there's some of
that with the 710s as well.

I just released a new benchmarking wrapper to measure seek performance
of drives and graph the result, and that's giving me interesting results
when comparing the 320 vs. traditional disk arrays.  I've attached a
teaser output from it on a few different drive setups I've tested
recently.  (The 320 test there was seeking against a smaller data set
than the regular arrays, but its performance doesn't degrade much based
on that anyway)

The program is at https://github.com/gregs1104/seek-scaling but it's
still quite rough to use.

--
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us

On 10/2/2011 10:35 PM, Greg Smith wrote:
>
> That sounds about the same performance as the 320 drive I tested
> earlier this year then.  You might try duplicating some of the
> benchmarks I ran on that:
> http://archives.postgresql.org/message-id/4D9D1FC3.4020207@2ndQuadrant.com
Thanks. Actually I had been looking for that email, which by brain
remembered but my computer and Google could not find ;)
>
> Make sure to reference the capacity of the drive though.  The 320
> units do scale their performance based on that, presumably there's
> some of that with the 710s as well.
This is a 100G drive. The performance specs vary curiously vs capacity
for the 710 series : write tps actually goes down as the size increases,
but bulk write data rate is higher for the larger drives.

I ran some pgbench earlier this evening, before reading your old email
above, so the parameters are different:

This is the new server with 100G 710 (AMD 6128 with 64G):

bash-4.1$ /usr/pgsql-9.1/bin/pgbench -T 600 -j 8 -c 64
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 8
duration: 600 s
number of transactions actually processed: 2182014
tps = 3636.520405 (including connections establishing)
tps = 3636.738290 (excluding connections establishing)

This is the output from iostat while the test runs:
Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s
avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00  4851.00    0.00 14082.00     0.00    73.76
10.73    45.72    3.25   0.05  71.60

This is our current production server type (Dual AMD 2346HE 32G 10K 300G
'raptor) with disk write cache turned off and with data and wal on the
same drive:

bash-3.2$ /usr/pgsql-9.1/bin/pgbench -T 600 -j 8 -c 64
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 8
duration: 600 s
number of transactions actually processed: 66426
tps = 108.925653 (including connections establishing)
tps = 108.941509 (excluding connections establishing)

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz
avgqu-sz   await  svctm  %util
sda               0.00   438.00  0.00 201.00     0.00     2.60
26.47    55.95  286.09   4.98 100.00

same server with disk write cache turned on:

bash-3.2$ /usr/pgsql-9.1/bin/pgbench -T 600 -j 8 -c 64
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 8
duration: 600 s
number of transactions actually processed: 184724
tps = 305.654008 (including connections establishing)
tps = 305.694317 (excluding connections establishing)

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz
avgqu-sz   await  svctm  %util
sda               0.00   668.00  0.00 530.00     0.00     4.55
17.57   142.99  277.28   1.89 100.10

There are some OS differences between the old and new servers : old is
running CentOS 5.7 while the new is running 6.0.
Old server has atime enabled while new has relatime mount option
specified. Both are running PG 9.1.1 from the yum repo.

One very nice measurement is the power consumption on the new server :
peak dissipation is 135W under the pgbench load
(measured on the ac input to the psu). Idle is draws around 90W.

On 10/2/2011 10:49 PM, David Boreham wrote:
>
> There are some OS differences between the old and new servers : old is
> running CentOS 5.7 while the new is running 6.0.
> Old server has atime enabled while new has relatime mount option
> specified. Both are running PG 9.1.1 from the yum repo.
>
>
Also the old server is using ext3 while the new has ext4 (with
discard/trim enabled).

Which repo did you get them from?

On Oct 2, 2011, at 10:24 PM, David Boreham wrote:

> On 10/2/2011 10:49 PM, David Boreham wrote:
>>
>> There are some OS differences between the old and new servers : old is running CentOS 5.7 while the new is running
6.0.
>> Old server has atime enabled while new has relatime mount option specified. Both are running PG 9.1.1 from the yum
repo.
>>
>>
> Also the old server is using ext3 while the new has ext4 (with discard/trim enabled).
>
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

On 10/2/2011 11:55 PM, Gregory Gerard wrote:
> Which repo did you get them from?
>
>
http://yum.postgresql.org/9.1/redhat/rhel-$releasever-$basearch