Re: [HACKERS] Need a builtin way to run all tests faster manner - Mailing list pgsql-hackers
| From | Andrew Dunstan |
|---|---|
| Subject | Re: [HACKERS] Need a builtin way to run all tests faster manner |
| Date | |
| Msg-id | f7de74f2-f7ca-de55-c476-1647fc132243@2ndQuadrant.com Whole thread Raw |
| In response to | Re: [HACKERS] Need a builtin way to run all tests faster manner (Tom Lane <tgl@sss.pgh.pa.us>) |
| Responses |
Re: [HACKERS] Need a builtin way to run all tests faster manner
(Andrew Dunstan <andrew.dunstan@2ndquadrant.com>)
|
| List | pgsql-hackers |
On 03/13/2017 12:35 AM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2017-03-11 22:14:07 -0500, Tom Lane wrote:
>>> This looks generally sane to me, although I'm not very happy about folding
>>> the "$(MKDIR_P) output_iso" call into pg_isolation_regress_check --- that
>>> seems weird and unlike the way it's done for the regular regression test
>>> case.
>> Yea, not super happy about that either - alternatively we could fold it
>> into pg_regress.
> Yeah, teaching pg_regress to auto-create the --temp-instance directory
> seems perfectly sane from here.
w.r.t. $subject, I thought it might be useful to get some recent stats
from the buildfarm. Results are below. The bin checks dwarf everything
else. Upgrade checks and isolation check are other items of significant
cost. Upgrade checks could be significantly shortened if we could avoid
rerunning the regression tests.
cheers
andrew
pgbfprod=> select s.branch, s.log_stage, count(*), avg(extract(epoch
from stage_duration)::numeric(15,2))::numeric(15,1),
stddev(extract(epoch from stage_duration)::numeric(15,2))::numeric(15,1)
from public.build_status_log s where sysname <> 'crake' and snapshot >
now() - interval '10 days' and log_stage !~ 'start|stop' group by
s.branch, s.log_stage having count(*) > 20 and avg(extract(epoch from
stage_duration)::numeric(15,2)) > 20 order by log_stage, branch <>
'HEAD', branch desc
; branch | log_stage | count
| avg | stddev
---------------+-----------------------------------------------+-------+--------+--------HEAD | bin-check.log
| 388 |
1739.0 | 1758.5REL9_6_STABLE | bin-check.log | 91 |
1430.5 | 1287.9REL9_5_STABLE | bin-check.log | 87 |
1140.0 | 994.1REL9_4_STABLE | bin-check.log | 72
| 751.0 | 666.8HEAD | check.log | 2305
| 263.1 | 1197.0REL9_6_STABLE | check.log | 610
| 294.7 | 1369.6REL9_5_STABLE | check.log | 627
| 170.1 | 819.6REL9_4_STABLE | check.log | 512
| 140.4 | 535.3REL9_3_STABLE | check.log | 449
| 112.0 | 446.0REL9_2_STABLE | check.log | 406
| 109.2 | 380.9HEAD | check-pg_upgrade.log | 1785
| 319.4 | 1310.5REL9_6_STABLE | check-pg_upgrade.log | 482
| 571.3 | 2811.0REL9_5_STABLE | check-pg_upgrade.log | 484
| 350.5 | 2160.3REL9_4_STABLE | check-pg_upgrade.log | 385
| 240.8 | 1278.9REL9_3_STABLE | check-pg_upgrade.log | 353
| 214.0 | 1188.3REL9_2_STABLE | check-pg_upgrade.log | 314
| 195.6 | 1016.6HEAD | config.log | 2216
| 84.5 | 101.5REL9_6_STABLE | config.log | 576
| 90.0 | 90.8REL9_5_STABLE | config.log | 584
| 114.0 | 358.8REL9_4_STABLE | config.log | 495
| 84.5 | 85.3REL9_3_STABLE | config.log | 431
| 97.9 | 100.7REL9_2_STABLE | config.log | 391
| 93.1 | 94.6HEAD | contrib-install-check-C.log | 2250
| 122.9 | 474.5REL9_6_STABLE | contrib-install-check-C.log | 606
| 124.6 | 410.3REL9_5_STABLE | contrib-install-check-C.log | 622
| 84.7 | 348.8REL9_4_STABLE | contrib-install-check-C.log | 508
| 105.9 | 434.1REL9_3_STABLE | contrib-install-check-C.log | 445
| 61.8 | 273.4REL9_2_STABLE | contrib-install-check-C.log | 403
| 54.3 | 205.7HEAD | contrib-install-check-cs_CZ.UTF-8.log | 184
| 25.7 | 11.8REL9_6_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 46
| 25.7 | 14.0REL9_5_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 42
| 23.8 | 16.6REL9_4_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 36
| 22.9 | 12.4HEAD | contrib-install-check-en_US.8859-15.log | 37
| 173.9 | 32.3HEAD | contrib-install-check-en_US.ISO8859-1.log | 33
| 244.7 | 35.7HEAD | contrib-install-check-en_US.log | 171
| 65.9 | 101.6REL9_6_STABLE | contrib-install-check-en_US.log | 42
| 61.7 | 89.3REL9_5_STABLE | contrib-install-check-en_US.log | 37
| 54.3 | 79.3REL9_4_STABLE | contrib-install-check-en_US.log | 33
| 53.7 | 71.5REL9_3_STABLE | contrib-install-check-en_US.log | 32
| 57.8 | 86.6REL9_2_STABLE | contrib-install-check-en_US.log | 29
| 46.9 | 63.8HEAD | contrib-install-check-en_US.utf8.log | 142
| 29.2 | 31.4REL9_6_STABLE | contrib-install-check-en_US.utf8.log | 30
| 40.2 | 43.7REL9_5_STABLE | contrib-install-check-en_US.utf8.log | 29
| 41.8 | 51.6REL9_4_STABLE | contrib-install-check-en_US.utf8.log | 25
| 56.6 | 75.8REL9_3_STABLE | contrib-install-check-en_US.utf8.log | 24
| 35.7 | 40.4REL9_2_STABLE | contrib-install-check-en_US.utf8.log | 21
| 31.3 | 36.1HEAD | contrib-install-check-en_US.UTF-8.log | 188
| 79.7 | 73.8REL9_6_STABLE | contrib-install-check-en_US.UTF-8.log | 36
| 78.2 | 68.6REL9_5_STABLE | contrib-install-check-en_US.UTF-8.log | 34
| 64.9 | 53.7REL9_4_STABLE | contrib-install-check-en_US.UTF-8.log | 30
| 68.4 | 56.2REL9_3_STABLE | contrib-install-check-en_US.UTF-8.log | 28
| 54.6 | 46.1REL9_2_STABLE | contrib-install-check-en_US.UTF-8.log | 25
| 48.3 | 39.8HEAD | contrib-install-check-POSIX.log | 33
| 227.7 | 42.6HEAD | ecpg-check.log | 2229
| 88.5 | 212.4REL9_6_STABLE | ecpg-check.log | 586
| 97.5 | 139.6REL9_5_STABLE | ecpg-check.log | 591
| 92.5 | 183.5REL9_4_STABLE | ecpg-check.log | 491
| 82.1 | 117.1REL9_3_STABLE | ecpg-check.log | 428
| 77.3 | 102.8REL9_2_STABLE | ecpg-check.log | 388
| 80.7 | 98.3REL9_6_STABLE | initdb-C.log | 606
| 21.2 | 73.1HEAD | initdb-en_US.8859-15.log | 37
| 241.8 | 24.8HEAD | initdb-en_US.ISO8859-1.log | 33
| 251.6 | 39.1HEAD | initdb-en_US.log | 171
| 54.3 | 103.8REL9_6_STABLE | initdb-en_US.log | 42
| 54.9 | 99.7REL9_5_STABLE | initdb-en_US.log | 37
| 61.2 | 112.5REL9_4_STABLE | initdb-en_US.log | 33
| 47.5 | 78.5REL9_3_STABLE | initdb-en_US.log | 32
| 46.0 | 74.6REL9_2_STABLE | initdb-en_US.log | 29
| 41.1 | 62.9HEAD | initdb-POSIX.log | 33
| 251.1 | 31.6HEAD | install-check-C.log | 2241
| 229.5 | 1227.9REL9_6_STABLE | install-check-C.log | 606
| 299.0 | 1601.9REL9_5_STABLE | install-check-C.log | 622
| 173.7 | 1147.2REL9_4_STABLE | install-check-C.log | 508
| 139.3 | 796.4REL9_3_STABLE | install-check-C.log | 446
| 104.7 | 713.3REL9_2_STABLE | install-check-C.log | 403
| 91.3 | 574.6HEAD | install-check-cs_CZ.ISO-8859-2.log | 137
| 37.5 | 1.5REL9_6_STABLE | install-check-cs_CZ.ISO-8859-2.log | 33
| 30.8 | 1.5REL9_5_STABLE | install-check-cs_CZ.ISO-8859-2.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-cs_CZ.ISO-8859-2.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-cs_CZ.ISO-8859-2.log | 24
| 24.9 | 1.2REL9_2_STABLE | install-check-cs_CZ.ISO-8859-2.log | 21
| 22.6 | 0.7HEAD | install-check-cs_CZ.UTF-8.log | 184
| 43.0 | 9.7REL9_6_STABLE | install-check-cs_CZ.UTF-8.log | 46
| 39.1 | 15.1REL9_5_STABLE | install-check-cs_CZ.UTF-8.log | 42
| 39.4 | 14.6REL9_4_STABLE | install-check-cs_CZ.UTF-8.log | 36
| 32.3 | 9.5REL9_3_STABLE | install-check-cs_CZ.UTF-8.log | 34
| 29.7 | 8.2REL9_2_STABLE | install-check-cs_CZ.UTF-8.log | 30
| 27.6 | 9.2HEAD | install-check-cs_CZ.WIN-1250.log | 137
| 37.3 | 1.2REL9_6_STABLE | install-check-cs_CZ.WIN-1250.log | 33
| 30.5 | 1.3REL9_5_STABLE | install-check-cs_CZ.WIN-1250.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-cs_CZ.WIN-1250.log | 25
| 26.3 | 0.7REL9_3_STABLE | install-check-cs_CZ.WIN-1250.log | 24
| 24.9 | 1.4REL9_2_STABLE | install-check-cs_CZ.WIN-1250.log | 21
| 22.3 | 0.6HEAD | install-check-de_DE.utf8.log | 129
| 35.4 | 9.4REL9_6_STABLE | install-check-de_DE.utf8.log | 22
| 25.8 | 7.8REL9_5_STABLE | install-check-de_DE.utf8.log | 21
| 27.4 | 8.0HEAD | install-check-en_US.8859-15.log | 37
| 136.9 | 21.2HEAD | install-check-en_US.ISO8859-1.log | 33
| 319.0 | 29.4HEAD | install-check-en_US.log | 171
| 65.7 | 58.7REL9_6_STABLE | install-check-en_US.log | 42
| 62.7 | 69.4REL9_5_STABLE | install-check-en_US.log | 37
| 59.4 | 56.2REL9_4_STABLE | install-check-en_US.log | 33
| 50.6 | 44.7REL9_3_STABLE | install-check-en_US.log | 32
| 50.2 | 46.8REL9_2_STABLE | install-check-en_US.log | 29
| 45.1 | 39.2HEAD | install-check-en_US.utf8.log | 142
| 47.6 | 40.3REL9_6_STABLE | install-check-en_US.utf8.log | 30
| 58.1 | 54.4REL9_5_STABLE | install-check-en_US.utf8.log | 29
| 61.7 | 57.4REL9_4_STABLE | install-check-en_US.utf8.log | 25
| 57.2 | 59.9REL9_3_STABLE | install-check-en_US.utf8.log | 24
| 51.3 | 47.7REL9_2_STABLE | install-check-en_US.utf8.log | 21
| 45.2 | 41.9HEAD | install-check-en_US.UTF-8.log | 188
| 153.5 | 162.7REL9_6_STABLE | install-check-en_US.UTF-8.log | 36
| 140.8 | 140.9REL9_5_STABLE | install-check-en_US.UTF-8.log | 34
| 145.7 | 144.3REL9_4_STABLE | install-check-en_US.UTF-8.log | 30
| 118.9 | 113.1REL9_3_STABLE | install-check-en_US.UTF-8.log | 28
| 107.0 | 101.8REL9_2_STABLE | install-check-en_US.UTF-8.log | 25
| 93.6 | 87.8HEAD | install-check-POSIX.log | 33
| 189.5 | 76.5HEAD | install-check-sk_SK.ISO-8859-2.log | 137
| 37.4 | 1.2REL9_6_STABLE | install-check-sk_SK.ISO-8859-2.log | 33
| 30.5 | 1.3REL9_5_STABLE | install-check-sk_SK.ISO-8859-2.log | 29
| 30.9 | 1.5REL9_4_STABLE | install-check-sk_SK.ISO-8859-2.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.ISO-8859-2.log | 24
| 24.8 | 1.2REL9_2_STABLE | install-check-sk_SK.ISO-8859-2.log | 21
| 22.7 | 0.6HEAD | install-check-sk_SK.UTF-8.log | 137
| 37.6 | 1.1REL9_6_STABLE | install-check-sk_SK.UTF-8.log | 33
| 31.0 | 1.8REL9_5_STABLE | install-check-sk_SK.UTF-8.log | 29
| 31.3 | 1.8REL9_4_STABLE | install-check-sk_SK.UTF-8.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.UTF-8.log | 24
| 25.0 | 1.2REL9_2_STABLE | install-check-sk_SK.UTF-8.log | 21
| 22.7 | 1.2HEAD | install-check-sk_SK.WIN-1250.log | 137
| 37.4 | 1.2REL9_6_STABLE | install-check-sk_SK.WIN-1250.log | 33
| 30.4 | 0.7REL9_5_STABLE | install-check-sk_SK.WIN-1250.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-sk_SK.WIN-1250.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.WIN-1250.log | 24
| 25.0 | 1.2REL9_2_STABLE | install-check-sk_SK.WIN-1250.log | 21
| 22.8 | 1.1HEAD | isolation-check.log | 2250
| 175.6 | 457.5REL9_6_STABLE | isolation-check.log | 606
| 201.0 | 527.7REL9_5_STABLE | isolation-check.log | 622
| 172.3 | 846.7REL9_4_STABLE | isolation-check.log | 508
| 146.0 | 602.3REL9_3_STABLE | isolation-check.log | 445
| 114.7 | 504.4REL9_2_STABLE | isolation-check.log | 403
| 57.2 | 300.6HEAD | make-contrib.log | 2176
| 43.7 | 67.6REL9_6_STABLE | make-contrib.log | 572
| 46.2 | 50.3REL9_5_STABLE | make-contrib.log | 579
| 44.9 | 54.2REL9_4_STABLE | make-contrib.log | 491
| 50.0 | 52.3REL9_3_STABLE | make-contrib.log | 428
| 48.6 | 51.4REL9_2_STABLE | make-contrib.log | 388
| 42.2 | 42.8HEAD | make-dist.log | 36
| 56.7 | 0.7HEAD | make-doc.log | 36
| 150.8 | 1.5REL9_2_STABLE | make-install.log | 403
| 22.0 | 29.8HEAD | make.log | 2328
| 363.1 | 581.0REL9_6_STABLE | make.log | 610
| 369.6 | 400.9REL9_5_STABLE | make.log | 627
| 392.8 | 556.8REL9_4_STABLE | make.log | 512
| 335.3 | 329.4REL9_3_STABLE | make.log | 449
| 320.0 | 311.0REL9_2_STABLE | make.log | 406
| 299.8 | 276.3HEAD | pl-install-check-en_US.8859-15.log | 37
| 32.9 | 3.3HEAD | pl-install-check-en_US.ISO8859-1.log | 33
| 34.3 | 5.1HEAD | pl-install-check-POSIX.log | 33
| 33.1 | 8.3REL9_2_STABLE | SCM-checkout.log | 406
| 33.2 | 87.8HEAD | sepgsql-test.log | 63
| 20.7 | 6.8HEAD | test-decoding-check.log | 1905
| 144.9 | 770.6REL9_6_STABLE | test-decoding-check.log | 493
| 226.3 | 1244.0REL9_5_STABLE | test-decoding-check.log | 495
| 147.1 | 1080.2REL9_4_STABLE | test-decoding-check.log | 419
| 168.1 | 1117.6HEAD | testmodules-install-check-C.log | 2233
| 22.6 | 42.3REL9_6_STABLE | testmodules-install-check-C.log | 602
| 25.6 | 35.8HEAD | testmodules-install-check-en_US.8859-15.log | 37
| 57.8 | 5.2HEAD | testmodules-install-check-en_US.ISO8859-1.log | 33
| 58.9 | 10.6HEAD | testmodules-install-check-en_US.log | 171
| 20.2 | 28.7REL9_6_STABLE | testmodules-install-check-en_US.utf8.log | 30
| 30.4 | 46.9HEAD | testmodules-install-check-en_US.UTF-8.log | 188
| 22.8 | 19.4REL9_6_STABLE | testmodules-install-check-en_US.UTF-8.log | 36
| 23.6 | 19.4REL9_5_STABLE | testmodules-install-check-en_US.UTF-8.log | 34
| 20.4 | 16.4HEAD | testmodules-install-check-POSIX.log | 33
| 72.1 | 14.7HEAD | typedefs.log | 96
| 74.8 | 99.1REL9_6_STABLE | typedefs.log | 28
| 106.3 | 119.7REL9_5_STABLE | typedefs.log | 28
| 100.0 | 111.7REL9_4_STABLE | typedefs.log | 24
| 88.1 | 100.7REL9_3_STABLE | typedefs.log | 23
| 85.6 | 95.1
(180 rows)
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
pgsql-hackers by date: