--- or in other words, greater variance in response times. With my simple understanding, that sounds like a problem for response-sensitive users.
If you need the throughput provided by 9.4, then using 9.3 gets lower variance simply be refusing to do 80% of the assigned work. If you don't need the throughput provided by 9.4, then you probably have some natural throttling in place.
If you want a real-world like test, you might try to crank up the -c and -j to the limit in 9.3 in a vain effort to match 9.4's performance, and see what that does to max latency. (After all, that is what a naive web app is likely to do--continue to make more and more connections as requests come in faster than they can finish.)
You're missing MauMau's point. In essence, he's comparing two systems with the same number of clients, issuing queries as fast as they can, and one can do 2000 TPS while the other one can do 10000 TPS. You would expect the lower-throughput system to have a *higher* average latency. Each query takes longer, that's why the throughput is lower. If you look at the avg_latency columns in the graphs (http://hlinnaka.iki.fi/xloginsert-scaling/padding/), that's exactly what you see.
But what MauMau is pointing out is that the *max* latency is much higher in the system that can do 10000 TPS. So some queries are taking much longer, even though in average the latency is lower. In an ideal, totally fair system, each query would take the same amount of time to execute, and after it's saturated, increasing the number of clients just makes that constant latency higher.
I thought that this was the point I was making, not the point I was missing. You have the same hard drives you had before, but now due to a software improvement you are cramming 5 times more stuff through them. Yeah, you will get bigger latency spikes. Why wouldn't you? You are now beating the snot out of your hard drives, whereas before you were not.
If you need 10,000 TPS, then you need to upgrade to 9.4. If you need it with low maximum latency as well, then you probably need to get better IO hardware as well (maybe not--maybe more tuning could help). With 9.3 you didn't need better IO hardware, because you weren't capable of maxing out what you already had. With 9.4 you can max it out, and this is a good thing.
If you need 10,000 TPS but only 2000 TPS are completing under 9.3, then what is happening to the other 8000 TPS? Whatever is happening to them, it must be worse than a latency spike.
On the other hand, if you don't need 10,000 TPS, than measuring max latency at 10,000 TPS is the wrong thing to measure.
