> The regression suite should reasonably be expected to pass on a system
> which has an "as good as it gets" installation. We haven't been very
> good about moving new features and fixed breakage _into_ the regression
> suite once it is fixed, but we shouldn't move known breakage into the
> regression suite.
If the queries which produce the problem are not exotic, then why
should they not go there? I would prefer a regression that took half
an hour or longer on modern hardware yet screened the code to a degree
that I could have reasonable faith in an installation if it passed the
regression. As it stands, I had to back off of a recommendation even
with a promise of my maintenance. I'll reiterate that quite common
queries are causing NULL pointers to get chased even in non-beta code.
When someone snoozes, our tests should alert us to that. I think to
ensure reliability, the regression must be as complete as possible.
It should work as an acceptance criterion. As it stands, it doesn't
test severely enough for much if any confidence in deployment.
> I'm not sure of the best way to document these kinds of problem
> statements. If we generate a "catchall file" of queries which crash, it
> runs the great risk of becoming stale and useless.
Even vaccinations require periodic renewal. The only degree to which
a test becomes stale is the degree that a feature is completely removed.
> though since it has been broken forever it doesn't count as a "must-fix"
> bug and may not make it into the v6.4 release. But it does count as a
> "should-fix" bug, and it's possible it could be fixed for v6.4...
Seg faults in the backend for standard queries would count as must-fix
from my view. The only justification for failure in the non-exotic
queries is that we are ignorant of the bug's existence. Once we are
aware, fixing these should become priority one.
\begin{soapbox}
At the very time ESR is lecturing the software community that open-
source is the way things should work, we have the opportunity to
support or refute his claims based on how seriously we take repair
and maintenance where non-exotic queries induce damage and failure.
Think about the free UNIX-like kernels. They are now gaining accept-
ance mostly because they keep running even when folk beat on them.
We have glass jaws, and it shows. Let's take the required steps to
firm them up.
\end{soapbox}
--
Christopher Oliver Traverse Internet
Systems Coordinator 223 Grandview Pkwy, Suite 108
oliver@traverse.net Traverse City, Michigan, 49684
"What good is a can of worms if you never open it?" -Bob Arning
