From: Jim Nasby [mailto:Jim.Nasby@BlueTreble.com] Sent: Wednesday, September 07, 2016 12:22 PM
On 9/4/16 7:34 AM, Mike Sofen wrote:
> You raise a good point. However, other disk activities involving
> large data (like backup/restore and pure large table copying), on both
> platforms, do not seem to support that notion. I did have both our IT
> department and Cisco turn on instrumentation for my last test,
> capturing all aspects of both tests on both platforms, and I’m hoping
> to see the results early next week and will reply again.
Something important to remember about Postgres is that it makes virtually no efforts to optimize IO; it throws the entire problem in the OSes lap. So differences in OS config or in IO *latency* can have a massive impact on performance. Because of the sensitivity to IO latency, you can also end up with a workload that only reports say 60% IO utilization but is essentially IO bound (would be 100% IO utilization if enough read-ahead was happening).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461
=============
Hi Jim,
Thanks for that info regarding the sensitivity to IO latency. As it turns out, our network guys have determined while the AWS Direct Connect pipe is running at “normal” speed, the end to latency is quite high and are working with AWS support to see if there are any optimizations to be done. To me, the performance differences have to be tied to networking, especially since it does appear that for these EC2 instances, all data – both SSD and network – is consuming bandwidth in their network “connection”, possibly adding to PG IO pressure. I’ll keep your note in mind as we evaluate next steps.
Mike
