Re: Any better plan for this query?..

From: Kevin Grittner
Subject: Re: Any better plan for this query?..
Date: ,
Msg-id: 4A0AB7BE.EE98.0025.0@wicourts.gov
(view: Whole thread, Raw)
In response to: Re: Any better plan for this query?..  (Dimitri)
List: pgsql-performance

Tree view

Any better plan for this query?..  (Dimitri, )
 Re: Any better plan for this query?..  (Craig Ringer, )
  Re: Any better plan for this query?..  (Dimitri, )
   Re: Any better plan for this query?..  (Chris, )
    Re: Any better plan for this query?..  (Dimitri, )
     Re: Any better plan for this query?..  (Richard Huxton, )
     Re: Any better plan for this query?..  (Craig Ringer, )
      Re: Any better plan for this query?..  (Kenneth Marshall, )
       Re: Any better plan for this query?..  (Matthew Wakeling, )
      Re: Any better plan for this query?..  (Dimitri, )
  Re: Any better plan for this query?..  (Kenneth Marshall, )
 Re: Any better plan for this query?..  (Heikki Linnakangas, )
  Re: Any better plan for this query?..  (Dimitri, )
   Re: Any better plan for this query?..  (Simon Riggs, )
    Re: Any better plan for this query?..  (Dimitri, )
     Re: Any better plan for this query?..  (Merlin Moncure, )
      Re: Any better plan for this query?..  (Dimitri, )
       Re: Any better plan for this query?..  (Simon Riggs, )
        Re: Any better plan for this query?..  (Dimitri, )
         Re: Any better plan for this query?..  (Tom Lane, )
          Re: Any better plan for this query?..  (Dimitri, )
          Re: Any better plan for this query?..  (Simon Riggs, )
         Re: Any better plan for this query?..  (Merlin Moncure, )
         Re: Any better plan for this query?..  (Simon Riggs, )
          Re: Any better plan for this query?..  (Dimitri, )
         Re: Any better plan for this query?..  (Aidan Van Dyk, )
          Re: Any better plan for this query?..  (Dimitri, )
           Re: Any better plan for this query?..  (Alvaro Herrera, )
            Re: Any better plan for this query?..  (Dimitri, )
             Re: Any better plan for this query?..  (Dimitri Fontaine, )
              Re: Any better plan for this query?..  (Dimitri, )
               Re: Any better plan for this query?..  (Simon Riggs, )
                Re: Any better plan for this query?..  (Dimitri, )
                 Re: Any better plan for this query?..  (Simon Riggs, )
               Re: Any better plan for this query?..  (Heikki Linnakangas, )
                Re: Any better plan for this query?..  (Dimitri, )
                 Re: Any better plan for this query?..  (Simon Riggs, )
                  Re: Any better plan for this query?..  (Dimitri, )
                   Re: Any better plan for this query?..  (Simon Riggs, )
                    Re: Any better plan for this query?..  (Dimitri, )
                     Re: Any better plan for this query?..  (Simon Riggs, )
                      Re: Any better plan for this query?..  ("Dave Dutcher", )
                       Re: Any better plan for this query?..  (Dimitri, )
                        Re: Any better plan for this query?..  (Merlin Moncure, )
                         Re: Any better plan for this query?..  (Dimitri, )
                          Re: Any better plan for this query?..  (Merlin Moncure, )
                           Re: Any better plan for this query?..  (Dimitri, )
                            Re: Any better plan for this query?..  (Merlin Moncure, )
                      Re: Any better plan for this query?..  (Dimitri, )
                       Re: Any better plan for this query?..  (Simon Riggs, )
                        Re: Any better plan for this query?..  (Dimitri, )
                         Re: Any better plan for this query?..  (Dimitri, )
                          Re: Any better plan for this query?..  (Simon Riggs, )
                           Re: Any better plan for this query?..  (Dimitri, )
                      Re: Any better plan for this query?..  (Tom Lane, )
                       Re: Any better plan for this query?..  (Dimitri, )
                       Re: Any better plan for this query?..  (Simon Riggs, )
                        Re: Any better plan for this query?..  (Matthew Wakeling, )
                         Re: Any better plan for this query?..  (Simon Riggs, )
                          Re: Any better plan for this query?..  (Simon Riggs, )
                          Re: Any better plan for this query?..  (Matthew Wakeling, )
                           Re: Any better plan for this query?..  (Simon Riggs, )
                           Re: Any better plan for this query?..  (Scott Carey, )
                          Re: Any better plan for this query?..  (Robert Haas, )
                        Re: Any better plan for this query?..  (Tom Lane, )
                         Re: Any better plan for this query?..  (Simon Riggs, )
                          Re: Any better plan for this query?..  (Tom Lane, )
                           Re: Any better plan for this query?..  (Robert Haas, )
                            Re: Any better plan for this query?..  (Simon Riggs, )
                             Re: Any better plan for this query?..  (Robert Haas, )
                              Re: Any better plan for this query?..  (Simon Riggs, )
                     Re: Any better plan for this query?..  (Scott Carey, )
                      Re: Any better plan for this query?..  (Dimitri, )
                       Re: Any better plan for this query?..  (Scott Carey, )
                        Re: Any better plan for this query?..  (Dimitri, )
                         Re: Any better plan for this query?..  (Scott Carey, )
                          Re: Any better plan for this query?..  (Dimitri, )
               Re: Any better plan for this query?..  (Simon Riggs, )
                Re: Any better plan for this query?..  (Dimitri, )
                 Re: Any better plan for this query?..  (Robert Haas, )
                  Re: Any better plan for this query?..  (Dimitri, )
                   Re: Any better plan for this query?..  ("Kevin Grittner", )
                   Re: Any better plan for this query?..  ("Joshua D. Drake", )
                    Re: Any better plan for this query?..  (Dimitri, )
                   Re: Any better plan for this query?..  (Robert Haas, )
                    Re: Any better plan for this query?..  (Dimitri, )
               Re: Any better plan for this query?..  (Stefan Kaltenbrunner, )
                Re: Any better plan for this query?..  (Dimitri, )
                 Re: Any better plan for this query?..  (Stefan Kaltenbrunner, )
                  Re: Any better plan for this query?..  (Matthew Wakeling, )
                   Re: Any better plan for this query?..  (Simon Riggs, )
                    Re: Any better plan for this query?..  (Matthew Wakeling, )
                     Re: Any better plan for this query?..  (Stefan Kaltenbrunner, )
                     Re: Any better plan for this query?..  (Tom Lane, )
                      Re: Any better plan for this query?..  (Robert Haas, )
                  Re: Any better plan for this query?..  (Dimitri, )
   Re: Any better plan for this query?..  ("Joshua D. Drake", )
           Re: Any better plan for this query?..  (Heikki Linnakangas, )
            Re: Any better plan for this query?..  (Dimitri, )
           Re: Any better plan for this query?..  (Andres Freund, )
            Re: Any better plan for this query?..  (Alvaro Herrera, )
             Re: Any better plan for this query?..  (Scott Carey, )
              Re: Any better plan for this query?..  (Dimitri, )
               Re: Any better plan for this query?..  (Scott Carey, )
                Re: Any better plan for this query?..  (Dimitri Fontaine, )
                 Re: Any better plan for this query?..  (Dimitri, )
                 Re: Any better plan for this query?..  (Simon Riggs, )
                Re: Any better plan for this query?..  (Dimitri, )
    Re: Any better plan for this query?..  (Gregory Stark, )
     Re: Any better plan for this query?..  (Simon Riggs, )
  Re: Any better plan for this query?..  (Matthew Wakeling, )
  Re: Any better plan for this query?..  (Tom Lane, )
   Re: Any better plan for this query?..  (Greg Stark, )
   Re: Any better plan for this query?..  (Robert Haas, )
  Re: Any better plan for this query?..  (Simon Riggs, )
   Re: Any better plan for this query?..  ("Joshua D. Drake", )
    Re: Any better plan for this query?..  (Aidan Van Dyk, )
     Re: Any better plan for this query?..  (Stephen Frost, )
   Re: Any better plan for this query?..  (Robert Haas, )
   Re: Any better plan for this query?..  (Glenn Maynard, )
    Re: Any better plan for this query?..  (Dimitri, )
    Re: Any better plan for this query?..  ("Kevin Grittner", )
     Re: Any better plan for this query?..  (Dimitri, )
      Re: Any better plan for this query?..  ("Kevin Grittner", )
  Re: Any better plan for this query?..  (Dimitri Fontaine, )
   Re: Any better plan for this query?..  (Alvaro Herrera, )
  Re: Any better plan for this query?..  (Robert Haas, )
   Re: Any better plan for this query?..  (Dimitri, )
 Re: Any better plan for this query?..  (Dimitri, )
  Re: Any better plan for this query?..  (Richard Huxton, )
  Re: Any better plan for this query?..  ("Albe Laurenz", )
   Re: Any better plan for this query?..  (Dimitri, )
    Re: Any better plan for this query?..  (Kenneth Marshall, )
 Re: Any better plan for this query?..  (Merlin Moncure, )
  Re: Any better plan for this query?..  (Merlin Moncure, )
   Re: Any better plan for this query?..  (Dimitri, )
    Re: Any better plan for this query?..  (Richard Huxton, )
     Re: Any better plan for this query?..  (Ries van Twisk, )
      Re: Any better plan for this query?..  (Dimitri, )
 Re: Any better plan for this query?..  (Kenneth Marshall, )
  Re: Any better plan for this query?..  (Dimitri, )
   Re: Any better plan for this query?..  (Kenneth Marshall, )
 Re: Any better plan for this query?..  (Dimitri, )
  Re: Any better plan for this query?..  (Alvaro Herrera, )
 Re: Any better plan for this query?..  (Simon Riggs, )

Dimitri <> wrote:
> The idea is good, but *only* pooling will be not enough. I mean if
> all what pooler is doing is only keeping no more than N backends
> working - it'll be not enough. You never know what exactly your
> query will do - if you choose your N value to be sure to not
> overload CPU and then some of your queries start to read from disk -
> you waste your idle CPU time because it was still possible to run
> other queries requiring CPU time rather I/O, etc...

I never meant to imply that CPUs were the only resources which
mattered.  Network and disk I/O certainly come into play.  I would
think that various locks might count.  You have to benchmark your
actual workload to find the sweet spot for your load on your hardware.
 I've usually found it to be around (2 * cpu count) + (effective
spindle count), where effective spindle count id determined not only
by your RAID also your access pattern.  (If everything is fully
cached, and you have no write delays because of a BBU RAID controller
with write-back, effective spindle count is zero.)

Since the curve generally falls off more slowly past the sweet spot
than it climbs to get there, I tend to go a little above the apparent
sweet spot to protect against bad performance in a different load mix
than my tests.

> I wrote some ideas about an "ideal" solution here (just omit the
> word "mysql" - as it's a theory it's valable for any db engine):
>
http://dimitrik.free.fr/db_STRESS_MySQL_540_and_others_Apr2009.html#note_5442

I've seen similar techniques used in other databases, and I'm far from
convinced that it's ideal or optimal.

-Kevin


pgsql-performance by date:

From: Dimitri Fontaine
Date:
Subject: Re: Any better plan for this query?..
From: Arjen van der Meijden
Date:
Subject: Re: AMD Shanghai versus Intel Nehalem