Thread: pgbench - compute & show latency consistently
Currently the latency is not computed and displayed consistently:
- the computation is wrong under -t (duration is zero...)
- depending on the conditions it is shown with a ":" syntax or
a "=" syntax.
The attached minor patch makes the computation & display more consistent.
--
Fabien.
Attachment
On 7/9/16 4:42 AM, Fabien COELHO wrote: > number of transactions per client: 1000 > number of transactions actually processed: 10000/10000 > -latency average = 15.844 ms > -latency stddev = 2.715 ms > +latency average: 15.844 ms > +latency stddev: 2.715 ms > tps = 618.764555 (including connections establishing) > tps = 622.977698 (excluding connections establishing) I think what you have here is that colons separate input parameters and equal signs separate result output. So I think it's OK the way it is. Maybe a better improvement would be introducing section headings like "test parameters" and "test results". -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hello Peter, >> number of transactions per client: 1000 >> -latency average = 15.844 ms >> +latency average: 15.844 ms >> tps = 618.764555 (including connections establishing) > > I think what you have here is that colons separate input parameters and > equal signs separate result output. So I think it's OK the way it is. Hmmm... Then other measures displayed are not all consistent with this theory. Also there is still the bug under -t which displays a 0 latency. The attached patch still fixes that and make it consistent the other way around, i.e. by using "=" for latency. I switched to use ":" for weight which is an input parameter. I let ":" when there is a long sentence to describe the figure displayed, more on aesthetical grounds. > Maybe a better improvement would be introducing section headings like > "test parameters" and "test results". This would add more lines to the report, to sure how desirable it is. -- Fabien.
Attachment
On Thus, July 7,2016 at 08:39 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote: > Also there is still the bug under -t which displays a 0 latency. Your patch clearly fixed the issue. > The attached patch still fixes that and make it consistent the other way > around, i.e. by using "=" for latency. I switched to use ":" for weight > which is an input parameter. I let ":" when there is a long sentence to > describe the figure displayed, more on aesthetic grounds. In the above context, I suggest few other changes. Present output-> progress: 1.0 s, 1221.0 tps, lat 0.816 ms stddev 0.272 Suggestion-> progress: 1.0 s, 1221.0 tps, lat avg 0.816 ms stddev 0.272 ms Present output-> SQL script 1: so.sql - weight = 1 (targets 50.0% of total) - 10010 transactions (50.1% of total, tps = 100.101872) - latencyaverage = 1.878 ms - latency stddev = 3.614 ms Suggestion-> SQL script 1: so.sql - weight = 1 (targets 50.0% of total) - 10010 transactions (50.1% of total) - tps = 100.101872 -latency average = 1.878 ms - latency stddev = 3.614 ms Apart from that, pgbench.sgml should be updated to reflect latency average in the output. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com
On 07/13/2016 11:39 AM, Fabien COELHO wrote: >>> number of transactions per client: 1000 >>> -latency average = 15.844 ms >>> +latency average: 15.844 ms >>> tps = 618.764555 (including connections establishing) >> >> I think what you have here is that colons separate input parameters and >> equal signs separate result output. So I think it's OK the way it is. > > Hmmm... Then other measures displayed are not all consistent with this > theory. > > Also there is still the bug under -t which displays a 0 latency. > > The attached patch still fixes that and make it consistent the other way > around, i.e. by using "=" for latency. I switched to use ":" for weight > which is an input parameter. I let ":" when there is a long sentence to > describe the figure displayed, more on aesthetical grounds. Committed, thanks! - Heikki
(I just pushed the patch, didn't see your post until after that) On 09/21/2016 01:07 PM, Kuntal Ghosh wrote: > On Thus, July 7,2016 at 08:39 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote: >> Also there is still the bug under -t which displays a 0 latency. > Your patch clearly fixed the issue. > >> The attached patch still fixes that and make it consistent the other way >> around, i.e. by using "=" for latency. I switched to use ":" for weight >> which is an input parameter. I let ":" when there is a long sentence to >> describe the figure displayed, more on aesthetic grounds. > In the above context, I suggest few other changes. > > Present output-> progress: 1.0 s, 1221.0 tps, lat 0.816 ms stddev 0.272 > Suggestion-> progress: 1.0 s, 1221.0 tps, lat avg 0.816 ms stddev 0.272 ms Yeah, perhaps. > Present output-> > SQL script 1: so.sql > - weight = 1 (targets 50.0% of total) > - 10010 transactions (50.1% of total, tps = 100.101872) > - latency average = 1.878 ms > - latency stddev = 3.614 ms > Suggestion-> > SQL script 1: so.sql > - weight = 1 (targets 50.0% of total) > - 10010 transactions (50.1% of total) > - tps = 100.101872 > - latency average = 1.878 ms > - latency stddev = 3.614 ms I think it fits well on a single line. > Apart from that, pgbench.sgml should be updated to reflect latency > average in the output. pgbench.sgml actually already had the "latency average = ..." version in its example. Even before this patch, we printed it with a "=" if one of options that caused per-transaction timings to be measured, like --rate, was used, and as ":" otherwise. - Heikki
On Wed, Sep 21, 2016 at 4:05 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > pgbench.sgml actually already had the "latency average = ..." version in its > example. Even before this patch, we printed it with a "=" if one of options > that caused per-transaction timings to be measured, like --rate, was used, > and as ":" otherwise. > > - Heikki > I'm talking about the first example in pgbench.sgml. Typical output from <application>pgbench</application> looks like: <screen> transaction type: <builtin: TPC-B (sort of)> scaling factor: 10 query mode: simple number of clients: 10 number of threads: 1 number of transactions per client: 1000 number of transactions actually processed: 10000/10000 tps = 85.184871 (including connections establishing) tps = 85.296346 (excluding connections establishing) </screen> Shouldn't we include latency average here as well and explain what it is? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com
Hello Kuntal, > <screen> > transaction type: <builtin: TPC-B (sort of)> > scaling factor: 10 > query mode: simple > number of clients: 10 > number of threads: 1 > number of transactions per client: 1000 > number of transactions actually processed: 10000/10000 > tps = 85.184871 (including connections establishing) > tps = 85.296346 (excluding connections establishing) > </screen> > > Shouldn't we include latency average here as well and explain what it is? Indeed, now it seems to be always printed but the documentation did not follow, there should be a: latency average = 117.392 ms In front of the tps line. Well, the performance displayed could also be improved... On my dual core SSD laptop I just got: sh> ./pgbench -c 10 -t 1000 starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 100 querymode: simple number of clients: 10 number of threads: 1 number of transactions per client: 1000 number of transactionsactually processed: 10000/10000 latency average = 9.527 ms tps = 1049.665115 (including connections establishing)tps = 1049.890194 (excluding connections establishing) Which is about 10 times better. -- Fabien.
On Wed, Sep 21, 2016 at 6:53 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote: > In front of the tps line. Well, the performance displayed could also be > improved... On my dual core SSD laptop I just got: > > sh> ./pgbench -c 10 -t 1000 > starting vacuum...end. > transaction type: <builtin: TPC-B (sort of)> > scaling factor: 100 > query mode: simple > number of clients: 10 > number of threads: 1 > number of transactions per client: 1000 > number of transactions actually processed: 10000/10000 > latency average = 9.527 ms > tps = 1049.665115 (including connections establishing) > tps = 1049.890194 (excluding connections establishing) > > Which is about 10 times better. Yes, you are right. In the documentation, the above example has not been updated across different pg versions. Although this is just an example, it should reflect the current performance. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com