Re: How to improve db performance with $7K?

From: Kevin Brown
Subject: Re: How to improve db performance with $7K?
Date: ,
Msg-id: 20050415020337.GD19518@filer
(view: Whole thread, Raw)
In response to: Re: How to improve db performance with $7K?  (Tom Lane)
Responses: Re: How to improve db performance with $7K?  (Alex Turner)
Re: How to improve db performance with $7K?  (Tom Lane)
Re: How to improve db performance with $7K?  (Vivek Khera)
List: pgsql-performance

Tree view

How to improve db performance with $7K?  (Steve Poe, )
 Re: How to improve db performance with $7K?  (Tom Lane, )
  Re: How to improve db performance with $7K?  (Steve Poe, )
 Re: How to improve db performance with $7K?  (Will LaShell, )
 Re: How to improve db performance with $7K?  ("Joshua D. Drake", )
  Re: How to improve db performance with $7K?  (Steve Poe, )
   Re: How to improve db performance with $7K?  (Bjoern Metzdorf, )
    Re: How to improve db performance with $7K?  (Josh Berkus, )
     Re: How to improve db performance with $7K?  (Steve Poe, )
      Re: How to improve db performance with $7K?  (Cott Lang, )
       Re: How to improve db performance with $7K?  (Steve Poe, )
        Re: How to improve db performance with $7K?  (Cott Lang, )
         Follow-Up: How to improve db performance with $7K?  (Steve Poe, )
          Re: Follow-Up: How to improve db performance with $7K?  (Thomas F.O'Connell, )
           Re: Follow-Up: How to improve db performance with $7K?  (Kevin Brown, )
            Re: Follow-Up: How to improve db performance with $7K?  ("Steinar H. Gunderson", )
            Re: Follow-Up: How to improve db performance with $7K?  (Thomas F.O'Connell, )
          Re: Follow-Up: How to improve db performance with $7K?  (Vivek Khera, )
           Re: Follow-Up: How to improve db performance with $7K?  (Will LaShell, )
     Re: How to improve db performance with $7K?  (Steve Poe, )
    Re: Follow-Up: How to improve db performance with $7K?  (PFC, )
     Re: Follow-Up: How to improve db performance with $7K?  ("Steinar H. Gunderson", )
   Re: How to improve db performance with $7K?  (PFC, )
    Re: How to improve db performance with $7K?  (Dave Cramer, )
     Re: How to improve db performance with $7K?  (Greg Stark, )
     Re: How to improve db performance with $7K?  (Alex Turner, )
      Re: How to improve db performance with $7K?  (Steve Poe, )
       Re: How to improve db performance with $7K?  (Alex Turner, )
        Re: How to improve db performance with $7K?  (Vivek Khera, )
         Re: How to improve db performance with $7K?  (Alex Turner, )
        Re: How to improve db performance with $7K?  (William Yu, )
         Re: How to improve db performance with $7K?  (Alex Turner, )
          Re: How to improve db performance with $7K?  (William Yu, )
           Re: How to improve db performance with $7K?  (Alex Turner, )
            Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
             Re: How to improve db performance with $7K?  (Alex Turner, )
              Re: How to improve db performance with $7K?  (Alex Turner, )
               Re: How to improve db performance with $7K?  (Alex Turner, )
              Re: How to improve db performance with $7K?  (Greg Stark, )
               Re: How to improve db performance with $7K?  (Alex Turner, )
               Re: How to improve db performance with $7K?  (Tom Lane, )
                Re: How to improve db performance with $7K?  (Kevin Brown, )
                 Re: How to improve db performance with $7K?  (Greg Stark, )
                  Re: How to improve db performance with $7K?  (Kevin Brown, )
                   Re: How to improve db performance with $7K?  (Tom Lane, )
                    Re: How to improve db performance with $7K?  (Kevin Brown, )
                     Re: How to improve db performance with $7K?  (Alex Turner, )
                      Re: How to improve db performance with $7K?  (PFC, )
                     Re: How to improve db performance with $7K?  (Tom Lane, )
                      Re: How to improve db performance with $7K?  (Kevin Brown, )
                       Re: How to improve db performance with $7K?  (Tom Lane, )
                        Re: How to improve db performance with $7K?  (Kevin Brown, )
                      Re: How to improve db performance with $7K?  (Greg Stark, )
                     Re: How to improve db performance with $7K?  (Vivek Khera, )
                      Re: How to improve db performance with $7K?  ("Joshua D. Drake", )
                       Re: How to improve db performance with $7K?  (Vivek Khera, )
                      Re: How to improve db performance with $7K?  (Kevin Brown, )
                    Re: How to improve db performance with $7K?  (Matthew Nuzum, )
                     Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
                   Re: How to improve db performance with $7K?  (Rosser Schwarz, )
                    Re: How to improve db performance with $7K?  (Kevin Brown, )
                   Re: How to improve db performance with $7K?  (Greg Stark, )
                   Re: How to improve db performance with $7K?  (Bruce Momjian, )
                    Re: How to improve db performance with $7K?  (Alex Turner, )
                     Re: How to improve db performance with $7K?  (Alvaro Herrera, )
                     Re: How to improve db performance with $7K?  (Bruce Momjian, )
              Re: How to improve db performance with $7K?  ("Douglas J. Trainor", )
               Re: How to improve db performance with $7K?  (, )
                Re: How to improve db performance with $7K?  (Alex Turner, )
                 Re: How to improve db performance with $7K?  (, )
            Re: How to improve db performance with $7K?  ("Douglas J. Trainor", )
            Re: How to improve db performance with $7K?  (PFC, )
             Re: How to improve db performance with $7K?  (Alan Stange, )
    Re: How to improve db performance with $7K?  (Dave Cramer, )
 Re: How to improve db performance with $7K?  ("Alexander Kirpa", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
 Re: How to improve db performance with $7K?  ("Matthew Nuzum", )
  Re: How to improve db performance with $7K?  (Greg Stark, )
   Re: How to improve db performance with $7K?  (PFC, )
  Re: How to improve db performance with $7K?  (Tom Lane, )
 Re: How to improve db performance with $7K?  ("Dave Held", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
  Re: How to improve db performance with $7K?  (Steve Poe, )
   Re: How to improve db performance with $7K?  ("Joshua D. Drake", )
   Re: How to improve db performance with $7K?  (William Yu, )
    Re: How to improve db performance with $7K?  (Greg Stark, )
     Re: How to improve db performance with $7K?  (Alex Turner, )
      Re: How to improve db performance with $7K?  (Greg Stark, )
       Re: How to improve db performance with $7K?  (Alex Turner, )
        Re: How to improve db performance with $7K?  (Jacques Caron, )
         Re: How to improve db performance with $7K?  (Alex Turner, )
          Re: How to improve db performance with $7K?  (Alex Turner, )
           Re: How to improve db performance with $7K?  (Jacques Caron, )
            Re: How to improve db performance with $7K?  (Alex Turner, )
          Re: How to improve db performance with $7K?  (Jacques Caron, )
         Re: How to improve db performance with $7K?  (Greg Stark, )
        Re: How to improve db performance with $7K?  (Alan Stange, )
        Re: How to improve db performance with $7K?  (John A Meinel, )
         Re: How to improve db performance with $7K?  (Alex Turner, )
          Re: How to improve db performance with $7K?  ("Joshua D. Drake", )
           Re: How to improve db performance with $7K?  (Alex Turner, )
        Re: How to improve db performance with $7K?  (Steve Poe, )
         Re: How to improve db performance with $7K?  (Alex Turner, )
     Re: How to improve db performance with $7K?  (Jacques Caron, )
      Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
     Re: How to improve db performance with $7K?  (William Yu, )
 Re: How to improve db performance with $7K?  ("Dave Held", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
 Re: How to improve db performance with $7K?  ("Dave Held", )
  Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
  Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
   Re: How to improve db performance with $7K?  (Alex Turner, )
 Re: How to improve db performance with $7K?  ("Dave Held", )
  Re: How to improve db performance with $7K?  (Alex Turner, )
   Re: How to improve db performance with $7K?  (William Yu, )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
  Re: How to improve db performance with $7K?  (Bruce Momjian, )
   Re: How to improve db performance with $7K?  (, )
    Re: How to improve db performance with $7K?  ("Jim C. Nasby", )
     What to do with 6 disks?  (Jeff Frost, )
      Re: What to do with 6 disks?  ("Jim C. Nasby", )
      Re: What to do with 6 disks?  (Josh Berkus, )
       Re: Opteron vs Xeon (Was: What to do with 6 disks?)  (Jeff Frost, )
        Re: Opteron vs Xeon (Was: What to do with 6 disks?)  ("J. Andrew Rogers", )
         Re: Opteron vs Xeon (Was: What to do with 6 disks?)  (Jeff Frost, )
          Re: Opteron vs Xeon (Was: What to do with 6 disks?)  (William Yu, )
        Re: Opteron vs Xeon (Was: What to do with 6 disks?)  (Vivek Khera, )
       Re: What to do with 6 disks?  (Vivek Khera, )
      Re: What to do with 6 disks?  (William Yu, )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
  Re: How to improve db performance with $7K?  (Bruce Momjian, )
  Re: How to improve db performance with $7K?  (Dawid Kuroczko, )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )
 Re: How to improve db performance with $7K?  ("Dave Held", )
 Re: How to improve db performance with $7K?  ("Mohan, Ross", )

Tom Lane wrote:
> Kevin Brown <> writes:
> > I really don't see how this is any different between a system that has
> > tagged queueing to the disks and one that doesn't.  The only
> > difference is where the queueing happens.  In the case of SCSI, the
> > queueing happens on the disks (or at least on the controller).  In the
> > case of SATA, the queueing happens in the kernel.
>
> That's basically what it comes down to: SCSI lets the disk drive itself
> do the low-level I/O scheduling whereas the ATA spec prevents the drive
> from doing so (unless it cheats, ie, caches writes).  Also, in SCSI it's
> possible for the drive to rearrange reads as well as writes --- which
> AFAICS is just not possible in ATA.  (Maybe in the newest spec...)
>
> The reason this is so much more of a win than it was when ATA was
> designed is that in modern drives the kernel has very little clue about
> the physical geometry of the disk.  Variable-size tracks, bad-block
> sparing, and stuff like that make for a very hard-to-predict mapping
> from linear sector addresses to actual disk locations.

Yeah, but it's not clear to me, at least, that this is a first-order
consideration.  A second-order consideration, sure, I'll grant that.

What I mean is that when it comes to scheduling disk activity,
knowledge of the specific physical geometry of the disk isn't really
important.  What's important is whether or not the disk conforms to a
certain set of expectations.  Namely, that the general organization is
such that addressing the blocks in block number order guarantees
maximum throughput.

Now, bad block remapping destroys that guarantee, but unless you've
got a LOT of bad blocks, it shouldn't destroy your performance, right?

> Combine that with the fact that the drive controller can be much
> smarter than it was twenty years ago, and you can see that the case
> for doing I/O scheduling in the kernel and not in the drive is
> pretty weak.

Well, I certainly grant that allowing the controller to do the I/O
scheduling is faster than having the kernel do it, as long as it can
handle insertion of new requests into the list while it's in the
middle of executing a request.  The most obvious case is when the head
is in motion and the new request can be satisfied by reading from the
media between where the head is at the time of the new request and
where the head is being moved to.

My argument is that a sufficiently smart kernel scheduler *should*
yield performance results that are reasonably close to what you can
get with that feature.  Perhaps not quite as good, but reasonably
close.  It shouldn't be an orders-of-magnitude type difference.



--
Kevin Brown                          


pgsql-performance by date:

From: Tom Lane
Date:
Subject: Re: How to improve db performance with $7K?
From: Kevin Brown
Date:
Subject: Re: How to improve db performance with $7K?