Hello Thomas,
> I committed the code change without the new TAP tests, because I
> didn't want to leave the open item hanging any longer.
Ok. Good.
> As for the test, ... [...]
Argh, so there are no tests that would have caught the regressions:-(
> ... I know it can fail, and your v18 didn't fix that, because...
>
> +check_pgbench_logs($bdir, '001_pgbench_log_1', $nthreads, 1, 3,
> ... this range can be exceeded.
Indeed. I meant to move that one in the TODO section as well, not just the
previous call, so that all time-sensitive tests are fully ignored but
reported, which would be enough for me.
> I suspect the number of aggregates could be made deterministic, as I
> described in an earlier message. What do you think about doing
> something like that first for the next release, before trying to add
> assertions about the number of aggregates?
I think that last time I did something to get more deterministic results
in pgbench, which involved a few lines of hocus-pocus in pgbench, the
patch got rejected:-)
An "ignored" tests looked like a good compromise to check how things are
going in the farm and to be able to check for more non regressions when
developing pgbench, without introducing behavioral changes.
> I'm with you on the importance of testing, but it seems better to start
> by making the thing more testable.
I'm used to my test patches being rejected, including modifying pgbench
behavior to make it more testable. Am I mad enough to retry? Maybe, maybe
not.
Attached the fully "ignored" version of the time features test as a patch.
--
Fabien.
