Re: Horizontal scalability/sharding - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Horizontal scalability/sharding
Date
Msg-id CA+TgmobquJziFm8nBUtF6hA9L_zMDxzq6_-jL-RTq2zpBysHFw@mail.gmail.com
Whole thread Raw
In response to Re: Horizontal scalability/sharding  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
Responses Re: Horizontal scalability/sharding  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Tue, Sep 1, 2015 at 2:04 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
> Memory bandwidth, for example. It's quite difficult to spot, because the
> intuition is that memory is fast, but thanks to improvements in storage (and
> stagnation in RAM bandwidth), this is becoming a significant issue.

I'd appreciate any tips on how to spot problems of this type.  But
it's my impression that perf, top, vmstat, and other Linux performance
tools will count time spent waiting for memory as CPU time, not idle
time.  If that's correct, that wouldn't explain workloads where CPU
utilization doesn't reach 100%.  Rather, it would show up as CPU time
hitting 100% while tps remains low.

> Process-management overhead is another thing we tend to ignore, but once you
> get to many processes all willing to work at the same time, you need to
> account for that.

Any tips on spotting problems in that area?

Thanks,

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Proposal: Implement failover on libpq connect level.
Next
From: Andres Freund
Date:
Subject: Re: Proposal: Implement failover on libpq connect level.