Andrew Dunstan writes:
> "Useful" is probably subjective. That list would at least be a good
> place to start, though. What combinations of variables do you think we
> would need?
First of all, I don't necessarily think that a large list of CPU/operation
system combinations is going to help much. IIRC, this round of platform
testing showed us two real problems, and both happened because the
operating system version in question came out the previous day, so we
could not have caught it. Much more problems arise when people use
different versions of secondary packages, such as Tcl, Perl, Kerberos,
Flex, Bison. So you would need to compile a large collection of these
things. The problem again is that it is usually the brand-new or the odd
intermediate version of such a tool that breaks things, so a "set and
forget" build farm is not going to catch it. Another real source of
problems are real systems. Weird combinations of packages, weird network
setups, weird applications, custom kernels. These cannot be detected on
out of the box setups. In fact, the regression tests might not detect
them at all.
Hence the open-source community approach. Closed-source development teams
can do all the above, with great effort. But by throwing out the code and
have real people test them on real systems with real applications, you can
do much better.
--
Peter Eisentraut peter_e@gmx.net
