Thread: ci: Improve macos startup using a cached macports installation
Hi, We have some issues with CI on macos and windows being too expensive (more on that soon in a separate email). For macos most of the obviously wasted time is spent installing packages with homebrew. Even with the package downloads being cached, it takes about 1m20s to install them. We can't just cache the whole homebrew installation, because it contains a lot of pre-installed packages. After a bunch of experimenting, I found a way to do this a lot faster: The attached patch installs macports and installs our dependencies from that. Because there aren't pre-existing macports packages, we can just cache the whole thing. Doing so naively wouldn't yield that much of a speedup, because it takes a while to unpack a tarball (or whatnot) with as many files as our dependencies have - that's a lot of filesystem metadata operations. Instead the script creates a HFS+ filesystem in a file and caches that - that's mounted within a few seconds. To further keep the size in check, that file is compressed with zstd in the cache. As macports has a package for IPC::Run and IO::Pty, we can use those instead of the separate cache we had for the perl installation. After the patch, the cached case takes ~5s to "install" and the cache is half the size than the one for homebrew. The comparison isn't entirely fair, because I used the occasion to not install 'make' (since meson is used for building) and llvm (we didn't enable it for the build anyway). That gain is a bit smaller without that, but still significant. An alternative implementation would be to create the "base" .dmg file outside of CI and download it onto the CI instances. But I didn't want to figure out the hosting situation for such files, so I thought this was the best near-medium term path. Greetings, Andres Freund
Attachment
Hi, On 2023-08-05 13:25:39 -0700, Andres Freund wrote: > We have some issues with CI on macos and windows being too expensive (more on > that soon in a separate email). For macos most of the obviously wasted time is > spent installing packages with homebrew. Even with the package downloads being > cached, it takes about 1m20s to install them. We can't just cache the whole > homebrew installation, because it contains a lot of pre-installed packages. > > After a bunch of experimenting, I found a way to do this a lot faster: The > attached patch installs macports and installs our dependencies from > that. Because there aren't pre-existing macports packages, we can just cache > the whole thing. Doing so naively wouldn't yield that much of a speedup, > because it takes a while to unpack a tarball (or whatnot) with as many files > as our dependencies have - that's a lot of filesystem metadata > operations. Instead the script creates a HFS+ filesystem in a file and caches > that - that's mounted within a few seconds. To further keep the size in check, > that file is compressed with zstd in the cache. > > As macports has a package for IPC::Run and IO::Pty, we can use those instead > of the separate cache we had for the perl installation. > > After the patch, the cached case takes ~5s to "install" and the cache is half > the size than the one for homebrew. > > The comparison isn't entirely fair, because I used the occasion to not install > 'make' (since meson is used for building) and llvm (we didn't enable it for > the build anyway). That gain is a bit smaller without that, but still > significant. > > > An alternative implementation would be to create the "base" .dmg file outside > of CI and download it onto the CI instances. But I didn't want to figure out > the hosting situation for such files, so I thought this was the best > near-medium term path. Given how significant an improvement this is in test time, and the limited blast radius, I am planning to push this soon, unless somebody opposes that soon. Greetings, Andres Freund
On 2023-08-19 11:47:33 -0700, Andres Freund wrote: > Given how significant an improvement this is in test time, and the limited > blast radius, I am planning to push this soon, unless somebody opposes that > soon. And done.