Re: Postgres bulk insert/ETL performance on high speed servers - test results - Mailing list pgsql-performance

From Jim Nasby
Subject Re: Postgres bulk insert/ETL performance on high speed servers - test results
Date
Msg-id d2676efc-d67a-4960-d07f-59d3db7588a2@BlueTreble.com
Whole thread Raw
In response to Re: Postgres bulk insert/ETL performance on high speed servers - test results  ("Mike Sofen" <msofen@runbox.com>)
Responses Re: Postgres bulk insert/ETL performance on high speed servers - test results  ("Mike Sofen" <msofen@runbox.com>)
List pgsql-performance
On 9/4/16 7:34 AM, Mike Sofen wrote:
> You raise a good point.  However, other disk activities involving large
> data (like backup/restore and pure large table copying), on both
> platforms, do not seem to support that notion.  I did have both our IT
> department and Cisco turn on instrumentation for my last test, capturing
> all aspects of both tests on both platforms, and I’m hoping to see the
> results early next week and will reply again.

Something important to remember about Postgres is that it makes
virtually no efforts to optimize IO; it throws the entire problem in the
OSes lap. So differences in OS config or in IO *latency* can have a
massive impact on performance. Because of the sensitivity to IO latency,
you can also end up with a workload that only reports say 60% IO
utilization but is essentially IO bound (would be 100% IO utilization if
enough read-ahead was happening).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461


pgsql-performance by date:

Previous
From: "Mkrtchyan, Tigran"
Date:
Subject: debug_assertions is enabled in official 9.6rc1 build
Next
From: "Mike Sofen"
Date:
Subject: Re: Postgres bulk insert/ETL performance on high speed servers - test results