Thread: ci: reduce macos test concurrency
Hi, The x86 mac VMs from cirrus-ci claim to have 12 CPUs, but when working on initdb caching for tests I noticed that using all those CPUs for tests hurts the test times noticably. See [1] (note that the overall time is influenced by different degrees of cache hit ratios): concurrency test time: 4 05:58 6 05:09 8 04:58 10 05:58 12 (current) 06:58 There's a fair bit of run-to-run variance, but the rough shape of these looks repeatable. I suspect the VMs might be overcommitted a fair bit - or macos just scales poorly. Cirrus-ci apparently is switching to M1 based macs, which could be related. It'd be good for us do that switch, as it'd give use ARM coverage for CI / cfbot. See also [3]. In 15 (and thus autoconf) the timings differ a bit less [2]: concurrency test time: 4 06:54 6 05:43 8 06:09 10 06:01 12 (current) 06:38 Looks like changing TEST_JOBS=6 or 8 would be a good idea. Greetings, Andres Freund [1] https://cirrus-ci.com/build/5254074546257920 [2] https://cirrus-ci.com/task/4888800445857792 [3] https://postgr.es/m/CAN55FZ2R%2BXufuVgJ8ew_yDBk48PgXEBvyKNvnNdTTVyczbQj0g%40mail.gmail.com
Hi, On 2022-09-26 21:02:08 -0700, Andres Freund wrote: > The x86 mac VMs from cirrus-ci claim to have 12 CPUs, but when working on > initdb caching for tests I noticed that using all those CPUs for tests hurts > the test times noticably. > > See [1] (note that the overall time is influenced by different degrees of > cache hit ratios): > > concurrency test time: > 4 05:58 > 6 05:09 > 8 04:58 > 10 05:58 > 12 (current) 06:58 > > There's a fair bit of run-to-run variance, but the rough shape of these > looks repeatable. > > I suspect the VMs might be overcommitted a fair bit - or macos just scales > poorly. Cirrus-ci apparently is switching to M1 based macs, which could be > related. It'd be good for us do that switch, as it'd give use ARM coverage for > CI / cfbot. See also [3]. > > > In 15 (and thus autoconf) the timings differ a bit less [2]: > > concurrency test time: > 4 06:54 > 6 05:43 > 8 06:09 > 10 06:01 > 12 (current) 06:38 > > Looks like changing TEST_JOBS=6 or 8 would be a good idea. Set it to 8 now. Greetings, Andres Freund