Re: Proposal of tunable fix for scalability of 8.4

From: Kevin Grittner
Subject: Re: Proposal of tunable fix for scalability of 8.4
Date: ,
Msg-id: 49B8E79C.EE98.0025.0@wicourts.gov
(view: Whole thread, Raw)
In response to: Re: Proposal of tunable fix for scalability of 8.4  (Grzegorz Jaśkiewicz)
Responses: Re: Proposal of tunable fix for scalability of 8.4  (Ron)
List: pgsql-performance

Tree view

Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
 Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
  Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
   Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
  Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
   Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
    Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
     Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
     Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
      Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
       Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
        Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
        Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
         Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
          Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
  Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
   Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
    Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
     Re: Proposal of tunable fix for scalability of 8.4  (decibel, )
      Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
       Re: Proposal of tunable fix for scalability of 8.4  (Josh Berkus, )
    Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
     Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
      Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
      Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
       Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
        Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
         Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
          Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
         Re: Proposal of tunable fix for scalability of 8.4  (decibel, )
        Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
        Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
         Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
          Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
           Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
         Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
         Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
          Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
           Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
            Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
          Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
           Re: Proposal of tunable fix for scalability of 8.4  (decibel, )
            Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
             Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
              Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
      Re: Proposal of tunable fix for scalability of 8.4  (Greg Smith, )
   Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
    Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
    Re: Proposal of tunable fix for scalability of 8.4  (Greg Smith, )
     Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
      Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
       Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
        Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
        Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
         Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
         Re: Proposal of tunable fix for scalability of 8.4  (, )
          Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
           Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
            Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
        Re: Proposal of tunable fix for scalability of 8.4  (Heikki Linnakangas, )
         Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
          Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
          Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
         Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
          Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
           Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
            Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
             Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
             Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
              Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
               Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
           Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
            Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
           Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
            Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
             Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
             Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
             Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
          Re: Proposal of tunable fix for scalability of 8.4  (Heikki Linnakangas, )
           Re: Proposal of tunable fix for scalability of 8.4  (Matthew Wakeling, )
       Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
        Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
         Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
          Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
           Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
            Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
             Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
              Re: Proposal of tunable fix for scalability of 8.4  (Alvaro Herrera, )
               Re: Proposal of tunable fix for scalability of 8.4  (Alvaro Herrera, )
                Re: Proposal of tunable fix for scalability of 8.4  (Alvaro Herrera, )
                 Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
                  Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
                   Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
           Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
         Re: Proposal of tunable fix for scalability of 8.4  (Bruce Momjian, )
          Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
           Re: Proposal of tunable fix for scalability of 8.4  (Bruce Momjian, )
         Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
      Re: Proposal of tunable fix for scalability of 8.4  (Greg Smith, )
     8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
      Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
      Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
       Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Alan Stange, )
        Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
         Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
          Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
           Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Robert Haas, )
            Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
            Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
           Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Bruce Momjian, )
            Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Bruce Momjian, )
      Re: 8.4 Performance improvements: was Re: Proposal of tunable fix for scalability of 8.4  (Greg Smith, )
   Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
    Re: Proposal of tunable fix for scalability of 8.4  (Grzegorz Jaśkiewicz, )
     Re: Proposal of tunable fix for scalability of 8.4  ("Kevin Grittner", )
      Re: Proposal of tunable fix for scalability of 8.4  (Ron, )
     Re: Proposal of tunable fix for scalability of 8.4  (Gregory Stark, )
      Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
    Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
     Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
      Re: Proposal of tunable fix for scalability of 8.4  (Alvaro Herrera, )
       Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
        Re: Proposal of tunable fix for scalability of 8.4  (decibel, )
      Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
    Re: Proposal of tunable fix for scalability of 8.4  (Tom Lane, )
     Re: Proposal of tunable fix for scalability of 8.4  (Scott Carey, )
    Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )
 Re: Proposal of tunable fix for scalability of 8.4  (Simon Riggs, )
  Re: Proposal of tunable fix for scalability of 8.4  ("Jignesh K. Shah", )

>>> Grzegorz Jaœkiewicz <> wrote:
> Scalability is something that is affected by everything, and fixing
> this makes sens as much as looking at possible fixes to make raids
> more scalable, which is looked at by someone else I think.
> So please, don't say that this doesn't make sense because he tested
it
> against ram disc. That was precisely the point of exercise.

I'm probably more inclined to believe that his change may have merit
than many here, but I can't accept anything based on this test until
someone answers the question, so far ignored by all responses, of
where the bottleneck is at the low end which allows linear scalability
up to 1000 users (which I assume means connections).

I'm particularly inclined to be suspicious of this test since my own
benchmarks, with real applications replaying real URL requests from a
production website that gets millions of hits per day, show that
response time and throughput are improved by using a connection pool
with queuing to limit the concurrent active queries.

My skepticism is not helped by the fact that in a previous discussion
with someone about performance as connections are increased, this
point was covered by introducing a "primitive" connection pool --
which used a one second sleep for a thread if the maximum number of
connections were already in use, rather than proper queuing and
semaphores.  That really gives no clue how performance would be with a
real connection pool.

-Kevin


pgsql-performance by date:

From: Laurent Laborde
Date:
Subject: Re: Full statement logging problematic on larger machines?
From: Scott Carey
Date:
Subject: Re: Proposal of tunable fix for scalability of 8.4