Chris:
There are a number of solutions on the market. A company called Soasta has a cloud based testing solution, my company has a product/solution that's highly customizable called StressWalk. On another computer I have a larger list with some ranking info I can send if you are interested.
Having been involved in a number of stress tests for customers, I think the most important questions in picking a service to help you get this done are:
- How much value am I likely to get by doing the testing? (If the app fails in production because your team missed something fundamental and it takes a day to fix via EIP, how much will that cost your company in lost revenue and intangibles like reputation?)
- How much money can I afford to spend?
- Do I have good instrumentation in place for my application, database, infrastructure, etc that I can easily correlate with the graphs produced by the stress testing application.
- Am I looking for a rubber stamp or real analysis of what's going on -- do I or my team have the time, expertise and experience in application, database and infrastructure performance optimization to do the analysis ourselves?
- Is my application UI difficult to stress test? (ie. Active X controls, Flex, Java Applet, SilverLight, Cross-site AJAX)
- To create a realistic load profile, do I need to simulate one user scenario or several?
- Does my application have user session and permission caching code? If yes, you may need to utilize a solution that can use large number of user credentials with different permission sets. If you do not have the users created already and it's a large enough population, you may want to use the stress testing service/tool to create and configure the users for your test in an automated way.
This should get you started. However, if you are leaning toward a service -- my analysis is that there are a few services priced under $1000 but they do not look very useful. The services we have found that look like they can provide some good value to their customers are in the $2500 - $15,000 range. For really complex stuff like application components interacting with mainframes and synchronized data extract and load to facilitate the work, you can expect things from the top end of that range up to $25-30K.
I'd be happy to have a conversation if you are serious about solving this problem.
-Jerry
Jerry Champlin
Absolute Performance Inc.
O: (303) 443-7000 x501
C: (303) 588-2547
jchamplin@absolute-performance.com
Chris wrote:
Hi,
Everyone says "load test using your app" - out of interest how does everyone do that at the database level?
I've tried playr (https://area51.myyearbook.com/trac.cgi/wiki/Playr) but haven't been able to get it working properly. I'm not sure what other tools are available.
TIA.
