Re: User concurrency thresholding: where do I look? - Mailing list pgsql-performance

Will try 16 and 32  CLOGBUFFER tomorrow:

But here is locks data again with about increased time profiling (about
2 minutes) for the connection with about 2000 users:

bash-3.00# time ./4_lwlock_waits.d 13583
^C

             Lock Id            Mode           Count
       ProcArrayLock          Shared               5
          XidGenLock       Exclusive              13
     CLogControlLock          Shared              14
          XidGenLock          Shared              15
     CLogControlLock       Exclusive              21
       WALInsertLock       Exclusive              77
        WALWriteLock       Exclusive             175
       ProcArrayLock       Exclusive             275

             Lock Id   Combined Time (ns)
          XidGenLock            194966200
       WALInsertLock            517955000
     CLogControlLock            679665100
        WALWriteLock           2838716200
       ProcArrayLock          44181002600


Top Wait time   seems to come from the following code path for
ProcArrayLock:

             Lock Id            Mode           Count
       ProcArrayLock       Exclusive              21

             Lock Id   Combined Time (ns)
       ProcArrayLock           5255937500

             Lock Id   Combined Time (ns)


              postgres`LWLockAcquire+0x1f0
              postgres`CommitTransaction+0x104
              postgres`CommitTransactionCommand+0xbc
              postgres`finish_xact_command+0x78
              postgres`exec_execute_message+0x42c
              postgres`PostgresMain+0x1838
              postgres`BackendRun+0x2f8
              postgres`ServerLoop+0x680
              postgres`PostmasterMain+0xda8
              postgres`main+0x3d0
              postgres`_start+0x17c
       5255937500



Regards,
Jignesh


Simon Riggs wrote:
> On Thu, 2007-07-26 at 15:44 -0400, Jignesh K. Shah wrote:
>
>> BEAUTIFUL!!!
>>
>> Using the Patch I can now go upto 1300 users without dropping.. But now
>> it still repeats at 1300-1350 users..
>>
>
> OK, can you try again with 16 and 32 buffers please?  We need to know
> how many is enough and whether this number needs to be variable via a
> parameter, or just slightly larger by default. Thanks.
>
>
>> I corrected the Lock Descriptions based on what I got from lwlock.h and
>> retried the whole scalability again with profiling again.. This time it
>> looks like the ProcArrayLock
>>
>
> That's what I would expect with that many users.
>
>
>>              Lock Id            Mode           Count
>>           XidGenLock       Exclusive               1
>>      CLogControlLock          Shared               2
>>           XidGenLock          Shared               2
>>         WALWriteLock       Exclusive               4
>>        WALInsertLock       Exclusive               8
>>      CLogControlLock       Exclusive               9
>>        ProcArrayLock       Exclusive               9
>>
>
> ...but as Tom mentioned, we need to do longer runs now so these counts
> get to somewhere in the hundreds so we have some statistical validity.
>
>