On Thu, 18 Jun 2026 11:24:04 +0800
Chao Li <li.evan.chao@gmail.com> wrote:
> Hi
>
> While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a question about the doc:
> ```
> <para>
> This option is useful when your custom script may raise errors
> such as unique constraint violations, but you want the benchmark
> to continue and measure performance including those failures.
> </para>
> ```
>
> The phrase “measure performance including those failures” gives the impression that failed transactions might be
countedin TPS, but they are not.
I agree with your point. The original phrasing "including those failures" is
indeed ambiguous and could mislead users into thinking failed transactions are
included in the TPS calculation.
> In my test, all transactions failed, and TPS was reported as 0:
> ```
> % pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f pgbench-coe.sql
> pgbench (19beta1)
> transaction type: pgbench-coe.sql
> scaling factor: 1
> query mode: simple
> number of clients: 1
> number of threads: 1
> maximum number of tries: 1
> number of transactions per client: 5
> number of transactions actually processed: 0/5
> number of failed transactions: 5 (100.000%)
> number of serialization failures: 0 (0.000%)
> number of deadlock failures: 0 (0.000%)
> number of other failures: 5 (100.000%)
> latency average = 0.389 ms (including failures)
> initial connection time = 4.172 ms
> tps = 0.000000 (without initial connection time)
> ```
>
> If this behavior is intended, I wonder if we should clarify the TPS counting logic more explicitly. For example:
> ```
> <para>
> This option is useful when your custom script may raise errors
> such as unique constraint violations, but you want the benchmark
> to continue despite individual statement failures. Failed transactions
> are reported separately and included in latency statistics, but are not
> counted as transactions actually processed for TPS.
> </para>
> ```
Your proposed documentation change is much clearer and accurately reflects
the actual behavior. I prefer this version as well.
Regards,
Yugo Nagata
--
Yugo Nagata <nagata@sraoss.co.jp>
