Re: CI and test improvements - Mailing list pgsql-hackers
From | Justin Pryzby |
---|---|
Subject | Re: CI and test improvements |
Date | |
Msg-id | 20221002213505.GE7745@telsasoft.com Whole thread Raw |
In response to | Re: CI and test improvements (Andres Freund <andres@anarazel.de>) |
Responses |
Re: CI and test improvements
(Andres Freund <andres@anarazel.de>)
Re: CI and test improvements (Andres Freund <andres@anarazel.de>) |
List | pgsql-hackers |
On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > Hi, > > On 2022-10-01 18:36:41 -0700, Andres Freund wrote: > > I am wondering if we should instead introduce a new "quickcheck" task that > > just compiles and runs maybe one test and have *all* other tests depend on > > that. Wasting a precious available windows instance to just fail to build or > > immediately fail during tests doesn't really make sense. > With a primed cache this takes ~32s, not too bad imo. 12s of that is > cloning the repo. Maybe - that would avoid waiting 4 minutes for a windows instance to start in the (hopefully atypical) case of a patch that fails in 1-2 minutes under linux/freebsd. If the patch were completely broken, the windows task would take ~4min to start, plus up to ~4min before failing to compile or failing an early test. 6-8 minutes isn't nothing, but doesn't seem worth the added complexity. Also, this would mean that in the common case, the slowest task would be delayed until after the SanityCheck task instance starts, compiles, and runs some test :( Your best case is 32sec, but I doubt that's going to be typical. I was thinking about the idea of cfbot handling "tasks" separately, similar to what it used to do with travis/appveyor. The logic for "windows tasks are only run if linux passes tests" could live there. That could also be useful if there's ever the possibility of running an additional OS on another CI provider, or if another provider can run windows tasks faster, or if we need to reduce our load/dependency on cirrus. I realized that goes backwards in some ways to the direction we've gone with cirrus, and I'm not sure how exactly it would do that (I suppose it might add ci-os-only tags to its commit message). > + # no options enabled, should be small > + CCACHE_MAXSIZE: "150M" Actually, tasks can share caches if the "cache key" is set. If there was a separate "Sanity" task, I think it should use whatever flags linux (or freebsd) use to avoid doing two compilations (with lots of cache misses for patches which modify *.h files, which would then happen twice, in serial). > + # use always: to continue after failures. Task that did not run count as a > + # success, so we need to recheck SanityChecks's condition here ... > - # task that did not run, count as a success, so we need to recheck Linux' > - # condition here ... Another/better justification/description is that "cirrus warns if the depending task has different only_if conditions than the dependant task". -- Justin
pgsql-hackers by date: