Thread: buildfarm issues
Now that we've been running for a while there are a few buildfarm issues that I need to address. First, do we keep the right data about the members? Essentially, we keep: <operating_system, os_version, compiler, compiler_version, architecture>. For Linux, we genarlly ask for the Distribution/distro-version instead of the OS/os-version. However, that lead to interesting situations - Gentoo for example is so "flexible" that in version 2004.03 you might easily be using kernel version 2.4.x or 2.6.x ... in fact it's almost impossible to tell what might be installed on a Gentoo system, or how it was compiled. So I'm really not sure how we should treat such systems. Second is the fact that systems change over time. People upgrade their machines. I'm considering a facility to allow people to change the <os-version,compiler-version> aspects of their registered personality - these will become essentially timestamped pieces of information, so we'll still be able to tie a set of values to a history item. Third, what can be done to improve the usefulness of the server / web site? I already have the following items on my list of TODOs: . allow sorting of member list by personality feature (e.g. see all the OS/X boxes together) . filter dashboard page to only show certain members (this already works but you have to construct the list manually, e.g. http://www.pgbuildfarm.org/cgi-bin/show_status.pl?member=cat&member=dog&member=loris&member=gibbon ) . an email alerting system for when a system changes state Fourth, there are some significant gaps in the buildfarm coverage. I am currently personally supplying the Windows and Cygwin members, but since that is on my laptop which is frequently required for other work it's not ideal, and there can be large gaps. Other gaps include HPUX, for example. If you run a machine that should be covered and can help, please do. After the initial setup there should be very little work involved. Finally, a volunteer or two to help me with both running and administering this would be great. Knowledhge of perl and Template Toolkit a decided advantage. Buildfarm was created to assist the principal hackers - so any and all (constructive) input is welcome. cheers andrew
On Friday 04 March 2005 10:11, Andrew Dunstan wrote: > Now that we've been running for a while there are a few buildfarm issues > that I need to address. > > First, do we keep the right data about the members? Essentially, we > keep: <operating_system, os_version, compiler, compiler_version, > architecture>. For Linux, we genarlly ask for the > Distribution/distro-version instead of the OS/os-version. However, that > lead to interesting situations - Gentoo for example is so "flexible" > that in version 2004.03 you might easily be using kernel version 2.4.x > or 2.6.x ... in fact it's almost impossible to tell what might be > installed on a Gentoo system, or how it was compiled. So I'm really not > sure how we should treat such systems. > > Second is the fact that systems change over time. People upgrade their > machines. I'm considering a facility to allow people to change the > <os-version,compiler-version> aspects of their registered personality - > these will become essentially timestamped pieces of information, so > we'll still be able to tie a set of values to a history item. What about using uname(1), cc -v, etc to glean this information and post it with each event logged? I belive you have all this stuff already in the config.log that is used already ? > > Third, what can be done to improve the usefulness of the server / web > site? I already have the following items on my list of TODOs: > . allow sorting of member list by personality feature (e.g. see all the > OS/X boxes together) > . filter dashboard page to only show certain members (this already works > but you have to construct the list manually, e.g. > http://www.pgbuildfarm.org/cgi-bin/show_status.pl?member=cat&member=dog&mem >ber=loris&member=gibbon ) > . an email alerting system for when a system changes state > > Fourth, there are some significant gaps in the buildfarm coverage. I am > currently personally supplying the Windows and Cygwin members, but since > that is on my laptop which is frequently required for other work it's > not ideal, and there can be large gaps. Other gaps include HPUX, for > example. If you run a machine that should be covered and can help, > please do. After the initial setup there should be very little work > involved. > > Finally, a volunteer or two to help me with both running and > administering this would be great. Knowledhge of perl and Template > Toolkit a decided advantage. > > Buildfarm was created to assist the principal hackers - so any and all > (constructive) input is welcome. > > cheers > > andrew > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org
Darcy Buskermolen wrote: >On Friday 04 March 2005 10:11, Andrew Dunstan wrote: > > >>Now that we've been running for a while there are a few buildfarm issues >>that I need to address. >> >>First, do we keep the right data about the members? Essentially, we >>keep: <operating_system, os_version, compiler, compiler_version, >>architecture>. For Linux, we genarlly ask for the >>Distribution/distro-version instead of the OS/os-version. However, that >>lead to interesting situations - Gentoo for example is so "flexible" >>that in version 2004.03 you might easily be using kernel version 2.4.x >>or 2.6.x ... in fact it's almost impossible to tell what might be >>installed on a Gentoo system, or how it was compiled. So I'm really not >>sure how we should treat such systems. >> >>Second is the fact that systems change over time. People upgrade their >>machines. I'm considering a facility to allow people to change the >><os-version,compiler-version> aspects of their registered personality - >>these will become essentially timestamped pieces of information, so >>we'll still be able to tie a set of values to a history item. >> >> > >What about using uname(1), cc -v, etc to glean this information and post it >with each event logged? I belive you have all this stuff already in the >config.log that is used already ? > > See previous para - on Linux we want the distro name and version, not "Linux" plus kernel version. uname doesn't seem to help much there. Also, I have no idea how portable cc -v is. Can we guarantee to have the compiler version properly identified on every platform? cheers andrew
Andrew, A couple of things, 1. we need to develop a matrix of systems/os/compiler to see what coverage we do have and compare it to the INSTALL guide. 2. the run_build.pl should be changed to keep the information on the system to date (and have the matrix in 1 change) 3. have the run_build.pl script check the build farm CVS to see if there is a new version of the build farm code and either download and/or alert the owner of the system I am able and willing to help out in all of the above. I am also willing to help fill the gap in systems if needed. I have been using perl for over 10 years now (since perl 4 days) and have been doing web/postgres coding for the last 6 years. Let me know what I can do to help Jim ---------- Original Message ----------- From: Andrew Dunstan <andrew@dunslane.net> To: Darcy Buskermolen <darcy@wavefire.com> Cc: PostgreSQL-development <pgsql-hackers@postgresql.org> Sent: Fri, 04 Mar 2005 14:28:09 -0500 Subject: Re: [HACKERS] buildfarm issues > Darcy Buskermolen wrote: > > >On Friday 04 March 2005 10:11, Andrew Dunstan wrote: > > > > > >>Now that we've been running for a while there are a few buildfarm issues > >>that I need to address. > >> > >>First, do we keep the right data about the members? Essentially, we > >>keep: <operating_system, os_version, compiler, compiler_version, > >>architecture>. For Linux, we genarlly ask for the > >>Distribution/distro-version instead of the OS/os-version. However, that > >>lead to interesting situations - Gentoo for example is so "flexible" > >>that in version 2004.03 you might easily be using kernel version 2.4.x > >>or 2.6.x ... in fact it's almost impossible to tell what might be > >>installed on a Gentoo system, or how it was compiled. So I'm really not > >>sure how we should treat such systems. > >> > >>Second is the fact that systems change over time. People upgrade their > >>machines. I'm considering a facility to allow people to change the > >><os-version,compiler-version> aspects of their registered personality - > >>these will become essentially timestamped pieces of information, so > >>we'll still be able to tie a set of values to a history item. > >> > >> > > > >What about using uname(1), cc -v, etc to glean this information and post it > >with each event logged? I belive you have all this stuff already in the > >config.log that is used already ? > > > > > > See previous para - on Linux we want the distro name and version, not > "Linux" plus kernel version. uname doesn't seem to help much there. > Also, I have no idea how portable cc -v is. Can we guarantee to have the > compiler version properly identified on every platform? > > cheers > > andrew > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq ------- End of Original Message -------
Andrew Dunstan <andrew@dunslane.net> writes:
> Also, I have no idea how portable cc -v is.
Not at all.
regards, tom lane
Andrew, > or 2.6.x ... in fact it's almost impossible to tell what might be > installed on a Gentoo system, or how it was compiled. So I'm really not > sure how we should treat such systems. Distribution, General OS, Kernel Version, Other Info e.g. SuSE Linux 2.6.8-7 64-Bit MS Windows XP SP2 Also, I think you should be recording the compile-time switches used on each machine and indexing them indivdually. I'd hate to find out that, for example, we'd broken --with-odbc and didn't know it because nobody in the buildfarm was using it. > Second is the fact that systems change over time. People upgrade their > machines. I'm considering a facility to allow people to change the > <os-version,compiler-version> aspects of their registered personality - > these will become essentially timestamped pieces of information, so > we'll still be able to tie a set of values to a history item. Yes ... actually, a quick (and automated) reminder e-mail once a month wouldn't hurt. And this machine registration info could be kept as a config file in the Buildfarm directory, for ease of update, and sent in with the buildfarm results. > Fourth, there are some significant gaps in the buildfarm coverage. I am I've a few responses from the PWN, such as a YDL machine on PPC4. I'll be passing these people along this week. > Finally, a volunteer or two to help me with both running and > administering this would be great. Knowledhge of perl and Template > Toolkit a decided advantage. I can help with the people end of things, but my web skills suck. -- Josh Berkus Aglio Database Solutions San Francisco
On Sun, Mar 06, 2005 at 21:12:03 -0800, Josh Berkus <josh@agliodbs.com> wrote: > > Also, I think you should be recording the compile-time switches used on each > machine and indexing them indivdually. I'd hate to find out that, for > example, we'd broken --with-odbc and didn't know it because nobody in the > buildfarm was using it. Another significant example is --enable-integer-datetimes.