Thread: Keeping track of buildfarm animals' personality
Way back in 2004 there was discussion of how we wanted to differentiate
buildfarm animals. We were flying blind to a large extent, as we didn't
have many similar things to compare to. We settled on {OS-name,
OS-version, Compiler-name, Compiler-version}. The idea was that the
names would be invariant but we would allow for OS and compiler upgrades.
There are two problems with this system, and have been since the get-go.
First, owners forget to run the upgrade-personalty.pl script when the
upgrade the OS and./or compiler. Second, there is no standardization of
the names. Complaints about these surfaced (again) at the Vancouver
unconference session about the buildfarm's future.
It was suggested that we should have the buildfarm client collect
information and report it in a piece of JSON which the server would then
use to update the personality table automatically. However, there are
some issues.
First there are issues of the source of information. And second there is
what information to extract from the source.
Let's take the simple case. On Linux this is fairly straightforward in
most cases. We can extract the "ID" and "VERSION_ID" attributes from
/etc/os-release, and we can get the compiler version with gcc--version
or clang --version, and get the version number out of the first line. I
think I'd strip out the vendor string, so it would just be something
like "13.2.1".
On windows, systeminfo gives you this sort of information:
OS Name: Microsoft Windows Server 2019 Datacenter
I guess by analogy with Linux we'd say that the OS Name is "Windows
Server" and the version just "2019".
cl /? shows you a version line we could similarly parse to get something
like 19.40.33808. We might want a way to tie that back to a particular
version of VS, but it would do for now.
For msys2 and cygwin we can get the version by calling "cygcheck
--version". These two are complicated by the fact that they are virtual
environments.
I don't have information about other OSs or compilers.
For now I'm intending just to collect the information and see what gaps
we have, before I start automating personality update.
If you have information about an OS or Compiler that's used in the
buildfarm that I haven't listed, please let me know.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Wed, Jun 12, 2024 at 4:13 AM Andrew Dunstan <andrew@dunslane.net> wrote:
> Way back in 2004 there was discussion of how we wanted to differentiate
> buildfarm animals. We were flying blind to a large extent, as we didn't
> have many similar things to compare to. We settled on {OS-name,
> OS-version, Compiler-name, Compiler-version}. The idea was that the
> names would be invariant but we would allow for OS and compiler upgrades.
>
> There are two problems with this system, and have been since the get-go.
> First, owners forget to run the upgrade-personalty.pl script when the
> upgrade the OS and./or compiler. Second, there is no standardization of
> the names. Complaints about these surfaced (again) at the Vancouver
> unconference session about the buildfarm's future.
I was sorry to miss that one! Basically every unconference was
unmissable, but you had to miss 3/4 of them due to physics...
> I don't have information about other OSs or compilers.
I wonder if most non-Linux, Unixoid systems would give a useful enough
answer with something like "uname -rs". It's a standard[1] after all.
The reason it's a bit useless on Linux is that Linux is [clears
throat, channels Richard Stallman] only a kernel. The interesting
information for Linux is the distro, but that doesn't apply for stuff
like FreeBSD, AIX, yada yada. Some results from systems near me right
now:
Darwin 23.5.0
FreeBSD 14.1-RELEASE
The Mac is a bit annoying because that's a kernel or Darwin version
(?), so you'd have to do some work to map it back to a macOS version
like 14.5... For eg FreeBSD you can also have the kernel out of sync
with the user space but normally you wouldn't so I think that's good
enough. Macs do have a way to get the OS version people are more
familiar with:
% sw_vers
ProductName: macOS
ProductVersion: 14.5
BuildVersion: 23F79
Maybe a uname-based default would be good enough for *BSD, Solaris
etc, and then we just have a small list of specialisations where we
know how to do better?
[1] https://pubs.opengroup.org/onlinepubs/009695299/utilities/uname.html
On Wed, Jun 12, 2024 at 8:29 AM Thomas Munro <thomas.munro@gmail.com> wrote: > For eg FreeBSD you can also have the kernel out of sync > with the user space but normally you wouldn't so I think that's good > enough. On second thoughts, perhaps we always want to capture the uname version, which is most likely about the kernel, and optionally also the userspace/distro version if we know how to with system-specific methods. That could be useful for diagnosing animals running in containers, which could come up on almost any OS as they all have container tech, and it's maybe even quite likely to be in use on systems on AIX, Solaris etc that people add to the farm from shared rare hardware.
Thomas Munro <thomas.munro@gmail.com> writes:
> On second thoughts, perhaps we always want to capture the uname
> version,
+1. Is there any reason not to capture "uname -a"? The
configure-based animals are effectively doing that already.
regards, tom lane
On 2024-Jun-12, Thomas Munro wrote: > On Wed, Jun 12, 2024 at 4:13 AM Andrew Dunstan <andrew@dunslane.net> wrote: > > There are two problems with this system, and have been since the get-go. > > First, owners forget to run the upgrade-personalty.pl script when the > > upgrade the OS and./or compiler. Second, there is no standardization of > > the names. Complaints about these surfaced (again) at the Vancouver > > unconference session about the buildfarm's future. > > I was sorry to miss that one! Basically every unconference was > unmissable, but you had to miss 3/4 of them due to physics... 2/3. Yeah, but I think the room selection algorithm we used is flawed. One that works better schedules all the most popular talks in the biggest room (matches what we did), but for the second room, we should sort in the opposite direction: put the fifth most popular topic in the last slot of the second room, and the sixth most popular in the last slot of the third room (instead, we filled the second and third room the same way as the first one). So for twelve topics it'd be like this, numbers represent topics ordered by popularity: room 1 room 2 room 3 1. 11. 12. 2. 9. 10. 3. 7. 8. 4. 5. 6. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/