Re: Cost limited statements RFC - Mailing list pgsql-hackers

From Kevin Grittner
Subject Re: Cost limited statements RFC
Date
Msg-id 1370726435.14514.YahooMailNeo@web162905.mail.bf1.yahoo.com
Whole thread Raw
In response to Re: Cost limited statements RFC  (Greg Smith <greg@2ndQuadrant.com>)
Responses Re: Cost limited statements RFC  (Greg Smith <greg@2ndQuadrant.com>)
List pgsql-hackers
Greg Smith <greg@2ndQuadrant.com> wrote:

> I suspect the reason we don't see as many complaints is that a
> lot more systems can handle 7.8MB/s of random reads then there
> are ones that can do 3.9MB/s of random writes.  If we removed
> that read limit, a lot more complaints would start rolling in
> about the read side.

I'll believe that all of that is true, but I think there's another
reason.  With multiple layers of write cache (PostgreSQL
shared_buffers, OS cache, controller or SSD cache) I think there's
a tendency for an "avalanche" effect.  Dirty pages stick to cache
at each level like snow on the side of a mountain, accumulating
over time.  When it finally breaks loose at the top, it causes more
from lower levels to break free as it passes.  The result at the
bottom can be devastating.

Before I leave the metaphor, I will admit that I've sometimes done
the equivalent of setting off an occasional stick of dynamite to
cause things to cascade down before they have built up to a more
dangerous level.

Setting aside the colorful imagery, with a write cache you often
see *very* fast writes for bursts or even sustained writes up to a
certain point, after which you suddenly have serious latency
spikes.  Reads tend to degrade more gracefully, giving you a sense
that you're starting to get into trouble while you still have time
to react to prevent extreme conditions.  At least that has been my
experience.  I think the "sudden onset" of problems from write
saturation contributes to the complaints.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Jeff Janes
Date:
Subject: Re: Cost limited statements RFC
Next
From: Kevin Grittner
Date:
Subject: Re: Batch API for After Triggers