Re: Allow a per-tablespace effective_io_concurrency setting - Mailing list pgsql-hackers

From Greg Stark
Subject Re: Allow a per-tablespace effective_io_concurrency setting
Date
Msg-id CAM-w4HMi0Sq8dAWQciFFGigpj5XB-MqnmBqZuRkzgOwtO_qXDg@mail.gmail.com
Whole thread Raw
In response to Re: Allow a per-tablespace effective_io_concurrency setting  (Merlin Moncure <mmoncure@gmail.com>)
Responses Re: Allow a per-tablespace effective_io_concurrency setting  (Bruce Momjian <bruce@momjian.us>)
Re: Allow a per-tablespace effective_io_concurrency setting  (Andres Freund <andres@anarazel.de>)
Re: Allow a per-tablespace effective_io_concurrency setting  (Merlin Moncure <mmoncure@gmail.com>)
List pgsql-hackers
On Thu, Sep 3, 2015 at 2:13 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
>   I find this talk of platters and spindles to be somewhat
> baroque; for a 200$ part I have to work pretty hard to max out the
> drive when reading and I'm still not completely sure if it's the drive
> itself, postgres, cpu, or sata interface bottlenecking me.  This will
> require a rethink of e_i_o configuration; in the old days there were
> physical limitations of the drives that were in the way regardless of
> the software stack but we are in a new era, I think.  I'm convinced
> prefetching works and we're going to want to aggressively prefetch
> anything and everything possible.  SSD controllers (at least the intel
> ones) are very smart.


Wouldn't SSDs need much *less* aggressive prefetching? There's still
latency and there are multiple I/O channels so they will still need
some. But spinning media gives latencies measured in milliseconds. You
can process a lot of tuples in milliseconds. If you have a hundred
spindles you want them all busy doing seeks because in the 5ms it
takes them to do that you can proess all the results on a single cpu
and the rest of time is spend waiting.

When your media has latency on the order of microseconds then you only
need to have a small handful of I/O requests in flight to keep your
processor busy.

-- 
greg



pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: Freeze avoidance of very large table.
Next
From: Bruce Momjian
Date:
Subject: Re: Allow a per-tablespace effective_io_concurrency setting