On 08/23/12 11:24 AM, Sébastien Lorion wrote:
> I think both kind of tests (general and app specific) are
> complementary and useful in their own way. At a minimum, if the
> general ones fail, why go to the expenses of doing the specific ones ?
> Setting up a meaningful application test can take a lot of time and it
> can be hard to pinpoint exactly where in the stack the performance
> drops occur. The way I see it, synthetic benchmarks allow to isolate
> somewhat the layers and serve as a base to validate application tests
> done later on. It surprises me that asking for the general perf
> behavior of a platform is controversial.
I don't use AWS at all. But, it shouldnt take more than a couple hours
to spin up an instance, populate a pgbench database and run a series of
pgbench runs against it, and do the same against any other sort of
system you wish to use as your reference.
I like to test with a database about twice the size of the available
memory if I'm testing IO, and I've found that pgbench -i -s ####, for
####=10000 it generates a 1 billion row table and uses about 150GB (and
a hour or so to initialize on fast IO hardware). I then run pgbench
with -c of about 2-4X the cpu/thread count, and -j of about -c/16, and a
-t of at least 10000 (so each client connection runs 10000 transactions).
on a modest but decent 2U class 2-socket dedicated server with a decent
raid card and raid10 across enough spindles, I can see numbers as high
as 5000 transactions/second with 15krpm rust, and 7000-8000 with a
couple MLC SSD's striped. trying to raid10 a bunch of SATA 7200 disks
gives numbers more like 1000. using host based raid, without a
write-back cache in the raid card, gives numbers about 1/2 the above.
the IOPS during these tests hit around 12000 or 15000 small writes/second.
doing this level of IO on a midsized SAN can often cause the SAN CPU to
run at 80%+ so if there's other activity on the SAN from other hosts,
good luck.
in a heavily virtualized shared-everything environment, I'm guessing
your numbers will be all over the place and difficult to achieve
consistency.
--
john r pierce N 37, W 122
santa cruz ca mid-left coast
