Re: using dbt2 postgresql 8.4 - rampup time issue - Mailing list pgsql-performance

On Fri, Jul 2, 2010 at 7:38 AM, MUHAMMAD ASIF <anaeem.it@hotmail.com> wrote:
> Hi,
>
> We are using dbt2 to check performance of postgresql 8.4 on Linux64 machine.
> When we increase "TERMINALS PER WAREHOUSE" TPM value increase rapidly but
> rampup time increase too , dbt2 estimated rampup time calculation do not
> work properly that’s why it run the test for wrong duration i.e.

A clarification of terms may help to start.  The "terminals per
warehouse" in the scripts correlates to the number terminals emulated.
 An emulated terminal is tied to a warehouse's district.  In other
words, the number of terminals translates to the number of districts
in a warehouse across the entire database.  To increase the terminals
per warehouse implies you have scaled the database differently, which
I'm assuming is not the case here.

> 1.
> Settings :
>     DATABASE CONNECTIONS: 50
>     TERMINALS PER WAREHOUSE: 10
>     SCALE FACTOR (WAREHOUSES): 200
>     DURATION OF TEST (in sec): 7200
> Result :
>                              Response Time (s)
>      Transaction      %    Average :    90th %        Total
> Rollbacks      %
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>         Delivery   3.96      0.285 :     0.023        26883
> 0   0.00
>        New Order  45.26      0.360 :     0.010       307335
> 3082   1.01
>     Order Status   3.98      0.238 :     0.003        27059
> 0   0.00
>          Payment  42.82      0.233 :     0.003       290802
> 0   0.00
>      Stock Level   3.97      0.245 :     0.002        26970
> 0   0.00
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>
>     2508.36 new-order transactions per minute (NOTPM)
>     120.1 minute duration
>     0 total unknown errors
>     2000 second(s) ramping up
>
> 2.
> Settings :
>     DATABASE CONNECTIONS: 50
>     TERMINALS PER WAREHOUSE: 40
>     SCALE FACTOR (WAREHOUSES): 200
>     DURATION OF TEST (in sec): 7200
> Result :
>                              Response Time (s)
>      Transaction      %    Average :    90th %        Total
> Rollbacks      %
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>         Delivery   3.95      8.123 :     4.605        43672
> 0   0.00
>        New Order  45.19     12.205 :     2.563       499356
> 4933   1.00
>     Order Status   4.00      7.385 :     3.314        44175
> 0   0.00
>          Payment  42.89      7.221 :     1.920       473912
> 0   0.00
>      Stock Level   3.97      7.093 :     1.887        43868
> 0   0.00
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>
>     7009.40 new-order transactions per minute (NOTPM)
>     69.8 minute duration
>     0 total unknown errors
>     8016 second(s) ramping up
>
> 3.
> Settings :
>     DATABASE CONNECTIONS: 50
>     TERMINALS PER WAREHOUSE: 40
>     SCALE FACTOR (WAREHOUSES): 200
>     DURATION OF TEST (in sec): 7200
> Result :
>                              Response Time (s)
>      Transaction      %    Average :    90th %        Total
> Rollbacks      %
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>         Delivery   3.98      9.095 :    16.103        15234
> 0   0.00
>        New Order  45.33      7.896 :    14.794       173539
> 1661   0.97
>     Order Status   3.96      8.165 :    13.989        15156
> 0   0.00
>          Payment  42.76      7.295 :    12.470       163726
> 0   0.00
>      Stock Level   3.97      7.198 :    12.520        15198
> 0   0.00
>     ------------  -----  ---------------------  -----------
> ---------------  -----
>
>     10432.09 new-order transactions per minute (NOTPM)
>     16.3 minute duration
>     0 total unknown errors
>     11227 second(s) ramping up
>
> These results show that dbt2 test actually did not run for 2 hours but it
> start varying with the increase of  "TERMINALS PER WAREHOUSE" value i.e. 1st
> Run ( 120.1 minute duration ), 2nd Run (69.8 minute duration) and 3rd Run
> (16.3 minute duration).

The ramp up times are actually as expected (explained below).  What
you are witnessing is more likely that the driver is crashing because
the values are out of range from the scale of the database.  You have
effectively told the driver that there are more than 10 districts per
warehouse, and have likely not built the database that way.  I'm
actually surprised the driver actually ramped up completely.

> To fix and sync with the rampup time, I have made a minor change in the
> dbt2-run-workload script i.e.
>
>     --- dbt2-run-workload      2010-07-02 08:18:06.000000000 -0400
>     +++ dbt2-run-workload   2010-07-02 08:20:11.000000000 -0400
>     @@ -625,7 +625,11 @@
>      done
>
>      echo -n "estimated rampup time: "
>     -do_sleep $SLEEP_RAMPUP
>     +#do_sleep $SLEEP_RAMPUP
>     +while ! grep START ${DRIVER_OUTPUT_DIR}/*/mix.log ; do
>     +       sleep 1
>     +done
>     +date
>      echo "estimated rampup time has elapsed"
>
>      # Clear the readprofile data after the driver ramps up.
>
> What is rempup time ? And what do you think about the patch?. Can you please
> guide me?. Thanks.

The ramp up time is supposed to be the multiplication of the terminals
per warehouse, the number of warehouses with the sleep time between
the creation of each terminal.  The only problem with your patch is
that the latest scripts (in the source code repo) breaks out the
client load into multiple instances of the driver program.  Thus there
is a log file per instance of the driver so your patch work work as
is.  Well, and there is that the ramp up calculation doesn't appear to
be broken. ;)

Regards,
Mark