Thread: CI, macports, darwin version problems
Hi, Problem #1: we're still using Ventura, but Cirrus has started doing this: Only ghcr.io/cirruslabs/macos-runner:sonoma is allowed. Automatically upgraded. It doesn't do it to cfbot, which runs macOS stuff on PGDG-hosted Mac Minis, but it does it to regular users who use free compute minutes tagged "instance:OSXCommunityInstance". This causes them to fail, because: [11:17:42.711] Error: Current platform "darwin 23" does not match expected platform "darwin 22" Sure enough, the sysinfo task shows "... Darwin Kernel Version 23.5.0...", but for cfbot it's still 22.y.z. So probably it's time to change to macOS 14 AKA sonoma AKA darwin 23. Problem #2: Once you do that with a simple s/ventura/sonoma/, it still "upgrades" to macos-runner:sonoma, which is not the same as macos-sonoma-base:latest. It has more versions of xcode installed? Not sure what else will break with that because I haven't successfully run it yet due to the next problem, but blind patch attached. Problem #3: If you have a macports installation cached (eg for CI in your github account), then the pre-existing macports installation will be for the wrong darwin version (error shown above). So I think we need to teach src/tools/ci/ci_macports_packages.sh to detect that condition and do a clean install. I can look into that, but does anyone already know how to do it? I know how to find out which darwin version is running: uname -r | sed 's/\..*//'. What I don't know is how to find the darwin version for a macports installation. I have found a terrible way to deduce it: sqlite3 /opt/local/var/macports/registry/registry.db "select max(os_major) from ports where os_major != 'any'" But that's stupid. There must be a way to ask it what version it was installed for ... I think it's the variable macports::os_major[2] (which is written in TCL, a language I can't follow too well), but I can't figure out where it's reading it from.... I hope there is a text file under /opt/local or at worst a SQLite database, or a way to ask the port command to spit that number out or ask it if it thinks migration is necessary... [1] https://github.com/cirruslabs/macos-image-templates/pkgs/container/macos-ventura-xcode [2] https://github.com/macports/macports-base/blob/bf27e0c98c7443877e081d5f6b6
Attachment
Thomas Munro <thomas.munro@gmail.com> writes:
> I know how to find out which darwin version is running: uname -r | sed
> 's/\..*//'. What I don't know is how to find the darwin version for a
> macports installation.
"port platform"?
regards, tom lane
On Wed, Jun 26, 2024 at 12:00 PM Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@gmail.com> writes: > > I know how to find out which darwin version is running: uname -r | sed > > 's/\..*//'. What I don't know is how to find the darwin version for a > > macports installation. > > "port platform"? Thanks, that's exactly what I was looking for. But I thought of an easier way: instead of trying to do my own cache invalidation with shell script and duct tape, I can include the current OS major version in the cache key used to carry the macports directory between CI runs. Hopefully Cirrus's cache machinery is smart enough to age out the old stuff eventually. This seems to have the desired effect. I've registered this thread to see how cfbot likes this, and see if anyone sees a problem with switching to the "macos-runner:sonoma" image, or the cache key scheme. https://commitfest.postgresql.org/48/5076/ FTR there is a newer macOS release that recently came out, Sequoia aka macOS 15, but the image available to us for CI is marked beta so I figured we can wait a bit longer for that.
Attachment
Thomas Munro <thomas.munro@gmail.com> writes: > But I thought of an easier way: instead of trying to do my own cache > invalidation with shell script and duct tape, I can include the > current OS major version in the cache key used to carry the > macports directory between CI runs. Hopefully Cirrus's cache machinery > is smart enough to age out the old stuff eventually. Sounds reasonable. > FTR there is a newer macOS release that recently came out, Sequoia aka > macOS 15, but the image available to us for CI is marked beta so I > figured we can wait a bit longer for that. Indeed not; that's only beta and will be so till September-ish. We don't really want to touch it yet because of this issue: https://www.postgresql.org/message-id/flat/CAMBWrQnEwEJtgOv7EUNsXmFw2Ub4p5P%2B5QTBEgYwiyjy7rAsEQ%40mail.gmail.com I'm not sure what the resolution of that will be, but we surely don't want to gate CI improvement on that. regards, tom lane
On Wed, Jun 26, 2024 at 4:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@gmail.com> writes: > > But I thought of an easier way: instead of trying to do my own cache > > invalidation with shell script and duct tape, I can include the > > current OS major version in the cache key used to carry the > > macports directory between CI runs. Hopefully Cirrus's cache machinery > > is smart enough to age out the old stuff eventually. > > Sounds reasonable. cfbot didn't like v2. It seems that github accounts using "instance:OSXCommunityInstance" are forced to use ghcr.io/cirruslabs/macos-runner:sonoma no matter what you ask for (example: [1]), while accounts configured to use user-supplied runners like the Mac Minis that cfbot is using *can't* use ghcr.io/cirruslabs/macos-runner:sonoma, and fail (example: [2]). I don't know why. So I think we should request ghcr.io/cirruslabs/macos-sonoma-base:latest. Personal github accounts will use macos-runner:sonoma instead, but at least it's the same OS release. Here's a new version like that, to see if cfbot likes it. Given that the OS release affects the macports_url we have to specify, I think this either means that we'll have to stay in sync with whatever macOS version is being forced for "instance:OSXCommunityInstance" users, or construct the macports_url automatically. Here is an attempt at the latter, as a second patch. Seems to work OK. For example, the setup_additional_packages step currently prints out: [06:23:08.584] macOS major version: 14 [06:23:09.672] MacPorts package URL: https://github.com/macports/macports-base/releases/download/v2.9.3/MacPorts-2.9.3-14-Sonoma.pkg As for the difference between the two types of image, they're described at [3]. The -runner images seem to be part of a project for faster starting VMs[4], which sounds like a pretty good reason to want to standardise on images to make pre-started instances fungible but there is perhaps also potential for selecting different xcode versions. > > FTR there is a newer macOS release that recently came out, Sequoia aka > > macOS 15, but the image available to us for CI is marked beta so I > > figured we can wait a bit longer for that. > > Indeed not; that's only beta and will be so till September-ish. > We don't really want to touch it yet because of this issue: > > https://www.postgresql.org/message-id/flat/CAMBWrQnEwEJtgOv7EUNsXmFw2Ub4p5P%2B5QTBEgYwiyjy7rAsEQ%40mail.gmail.com > > I'm not sure what the resolution of that will be, but we surely > don't want to gate CI improvement on that. Urgh. Also we have to wait for MacPorts to make a release for Sequoia, which might involve lots of maintainers hunting stuff like that. (If Cirrus starts forcing people to use Sequoia before then, that'd be a problem.) [1] https://cirrus-ci.com/task/4747151899623424 [2] https://cirrus-ci.com/task/6601239016767488 [3] https://github.com/cirruslabs/macos-image-templates [4] https://cirrus-runners.app/blog/2024/04/11/optimizing-startup-time-of-cirrus-runners/