Hi,
I noticed that CI tasks on the postgres' github repo fail for some branches on
macos [1].
Initially I thought the problem was related to some outdated cache of the
macports installation. And indeed clearing that out does fix the issue - but
only temporarily and fixing one branch would cause some others to fail.
A fair bit of debugging later I realized that the problem is due to
src/tools/ci/ci_macports_packages.sh not actually installing missing packages
unless starting without a cache.
The cache key for the macports installation currently does not include the
major version. However, the md5 of src/tools/ci/ci_macports_packages.sh is
included in the cache key, and 17+ have a typo fix, leading to a different
cache key.
Whenever the cache was built with 15, 16 would fail, because gmake was
installed but not meson. And vice versa. It gets worse, see further down.
While we could fix the issue by including the major version in the key, that'd
only be a partial fix, because the goal is to be able to adjust the list of
packages.
The reason src/tools/ci/ci_macports_packages.sh does not "incrementally"
install new packages anymore is that port setrequested changed it's error
behaviour:
port setrequested $package errors out if $package is not installed. In the
past this was also true if multiple packages were passed in. However, now an
error is only raised if the first package is not installed:
andres@m4-dev ~ % sudo port -v setrequested bzip2 libiconv non-existing-package; echo $?
Setting requested flag for bzip2 @1.0.8_0 to 1
Setting requested flag for libiconv @1.17_0 to 1
0
andres@m4-dev ~ % sudo port -v setrequested non-existing-package bzip2 libiconv; echo $?
Error: non-existing-package is not installed
1
This means that src/tools/ci/ci_macports_packages.sh would only enter the
install-a-missing-package path if the first mentioned package was missing. As
the first package did not change between 15 and 16, we'd never install the
missing gmake/meson.
What's worse, because the remove-unneeded-packages path *does* work, we
eventually end up with a cache that has neither gmake nor meson installed
causing both 15 and 16 to fail.
The easiest fix I can see is to simply loop over the to-be-installed installed
packages and mark them as installed one-by-one. That's a few seconds slower,
but that's not too bad. Anyone got a better idea?
I'll try to report a bug to macports, but I suspect we ought to fix this for
CI before this is addressed via a new macports release.
Greetings,
Andres Freund
[1] https://cirrus-ci.com/github/postgres/postgres/
