Thread: block device benchmarking

block device benchmarking

From
Markus Wanner
Date:
Hi,

I'm fiddling with a hand-made block device based benchmarking thingie,
which I want to run random reads and writes of relatively small blocks
(somewhat similar to databases). I'm much less interested in measuring
throughput, but rather in latency. Besides varying block sizes, I'm also
testing with a varying number of concurrent threads and varying
read/write ratios. As a result, I'm interested in roughly the following
graphs:

 * (single thread) i/o latency vs. seek distance
 * (single thread) throughput vs. (accurator) position
 * (single thread) i/o latency vs. no of concurrent threads
 * total requests per second + throughput vs. no of concurrent threads
 * total requests per second + throughput vs. read/write ratio
 * total requests per second + throughput vs. block size
 * distribution of access times (histogram)

(Of course, not all of these are relevant for all types of storages.)

Does there already exist a tool giving (most of) these measures? Am I
missing something interesting? What would you expect from a block device
benchmarking tool?

Regards

Markus Wanner

Re: block device benchmarking

From
"M. Edward (Ed) Borasky"
Date:
Markus Wanner wrote:
> Hi,
>
> I'm fiddling with a hand-made block device based benchmarking thingie,
> which I want to run random reads and writes of relatively small blocks
> (somewhat similar to databases). I'm much less interested in measuring
> throughput, but rather in latency. Besides varying block sizes, I'm also
> testing with a varying number of concurrent threads and varying
> read/write ratios. As a result, I'm interested in roughly the following
> graphs:
>
>  * (single thread) i/o latency vs. seek distance
>  * (single thread) throughput vs. (accurator) position
>  * (single thread) i/o latency vs. no of concurrent threads
>  * total requests per second + throughput vs. no of concurrent threads
>  * total requests per second + throughput vs. read/write ratio
>  * total requests per second + throughput vs. block size
>  * distribution of access times (histogram)
>
> (Of course, not all of these are relevant for all types of storages.)
>
> Does there already exist a tool giving (most of) these measures? Am I
> missing something interesting? What would you expect from a block device
> benchmarking tool?
>
> Regards
>
> Markus Wanner
>

Check out the work of Jens Axboe and Alan Brunelle, specifically the
packages "blktrace" and "fio". "blktrace" acts as a "sniffer" for I/O,
recording the path of every I/O operation through the block I/O layer.
Using another tool in the package, "btreplay/btrecord", you can
translate the captured trace into a benchmark that re-issues the I/Os.
And the third tool in the package, "btt", does statistical analysis. I
don't think you really need "benchmarks" if you can extract this kind of
detail from a real application. :)

However, if you do want to build a benchmark, "fio" is a customizable
benchmark utility. In the absence of real-world traces, you can emulate
any I/O activity pattern with "fio". "fio" is what Mark Wong's group has
been using to characterize filesystem behavior. I'm not sure where the
presentations are at the moment, but there is some of it at

http://wiki.postgresql.org/wiki/HP_ProLiant_DL380_G5_Tuning_Guide

There are also some more generic filesystem benchmarks like "iozone" and
"bonnie++". They're a good general tool for comparing filesystems and
I/O subsystems, but the other tools are more useful if you have a
specific workload, for example, a PostgreSQL application.

BTW ... I am working on my blktrace howto even as I type this. I don't
have an ETA -- that's going to depend on how long it takes me to get the
PostgreSQL benchmarks I'm using to work on my machine. But everything
will be on Github at

http://github.com/znmeb/linux_perf_viz/tree/master/blktrace-howto

as it evolves.

Re: block device benchmarking

From
Markus Wanner
Date:
Hi,

M. Edward (Ed) Borasky wrote:
> Check out the work of Jens Axboe and Alan Brunelle, specifically the
> packages "blktrace" and "fio".

Thank you for these pointers, that looks pretty interesting.

> There are also some more generic filesystem benchmarks like "iozone" and
> "bonnie++".

Those are targeted at filesystems, which hide some of the disk's
characteristics I'm interested in. But thanks.

> BTW ... I am working on my blktrace howto even as I type this. I don't
> have an ETA -- that's going to depend on how long it takes me to get the
> PostgreSQL benchmarks I'm using to work on my machine. But everything
> will be on Github at
>
> http://github.com/znmeb/linux_perf_viz/tree/master/blktrace-howto
>
> as it evolves.

Cool, looking forward to reading that.

Regards

Markus Wanner

Re: block device benchmarking

From
"M. Edward (Ed) Borasky"
Date:
Markus Wanner wrote:
> M. Edward (Ed) Borasky wrote:
>> Check out the work of Jens Axboe and Alan Brunelle, specifically the
>> packages "blktrace" and "fio".
>> BTW ... I am working on my blktrace howto even as I type this. I don't
>> have an ETA -- that's going to depend on how long it takes me to get the
>> PostgreSQL benchmarks I'm using to work on my machine. But everything
>> will be on Github at
>>
>> http://github.com/znmeb/linux_perf_viz/tree/master/blktrace-howto
>>
>> as it evolves.
>
> Cool, looking forward to reading that.

I've got it about half done at this point. What's still to be done is
build the PostgreSQL example and the specialized post-processing code.
But the basics -- how to build "blktrace" from source and run it -- are
there. It will probably be next weekend before I get a chance to build
the PostgreSQL example.

http://github.com/znmeb/linux_perf_viz/raw/master/blktrace-howto/blktrace-howto.pdf

If anyone wants to comment on it, the best way might be to join Github
and send me in-project messages there. I've got some documentation
questions I need to ask on the blktrace mailing list.
--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P), WOM

I've never met a happy clam. In fact, most of them were pretty steamed.