Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement) - Mailing list pgsql-hackers

From Fabien COELHO
Subject Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)
Date
Msg-id alpine.DEB.2.02.1307171337260.3991@localhost6.localdomain6
Whole thread Raw
In response to Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)  (Tatsuo Ishii <ishii@postgresql.org>)
Responses Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)
List pgsql-hackers
>> tps = 9818.741060 (including connections establishing)
>>
>> So I thought I could squeeze 10000 TPS from my box.
>> Then I tried with -R 5000 tps.
>>
>> number of transactions actually processed: 1510640
>> average rate limit lag: 0.304 ms (max 19.101 ms)
>> tps = 5035.409397 (including connections establishing)
>>
>> As you can see, I got about 5000 tps as expected.

Yep, it works:-)

>> But I'm confused by the lag:
>>
>> 0.304 ms * 1510640 = 459.2 seconds, which is longer than 300 seconds
>> (specified by -T). Am I missing something?

The lag is reasonnable, althought no too good. One transaction is about 
1.2 ms, the lag is much smaller than that, and you are at about 50% of the 
maximum load. I've got similar figures on my box for such settings. It 
improves if your reduce the number of clients.

If you reduce the number of clients, or add more threads, the lag is 
reduced.

> BTW, the system was Linux (kernel 3.0.77).

> tps = 6730.940132 (including connections establishing)
> $ pgbench -S -n -c 10 -T 10  -R 3000 test

> average rate limit lag: 0.089 ms (max 27.301 ms)
> tps = 2983.707895 (including connections establishing)
>
> 0.089 ms * 29840 = 2.66 seconds. Not too bad compared with 10
> seconds.

Indeed, that is better. Transactions are about 1.5 ms and you run at about 
45% of the maximum load here.

> On Linux maybe the overhead to calculate the lag is bigger
> than Mac OS X? Just my wild guess though...

I would be surprised that this would be the issue is to compute the 
measure, compared to network connections and the like. With -S the bench 
is cpu bound. Possibly a better scheduler/thread management on OSX? Or 
more available cores? Well, I do not know! At high load with clients 
running on the same box as the server, and with more clients & server than 
available cores, there is a lot of competition between processes, and 
between clients that share a unique thread, and a log context switching 
whoch will result in a measured lag.

-- 
Fabien.



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Improvement of checkpoint IO scheduler for stable transaction responses
Next
From: Michael Paquier
Date:
Subject: Re: Adding optionally commit number in PG_VERSION_STR