Thread: Should I run regression tests?
Hi, I saw the file INSTALL and I ran regression tests, I saw lots of failed tests, what do they mean? If I have failed tests, how do I fix it? If I don't run regression tests, what's the consequences I have? Would anyone tell me, thanks. Best regards, Doug.
On Mon, 8 Jun 1998, Doug Lo wrote:
> Hi,
>
> I saw the file INSTALL and I ran regression tests, I saw lots of failed tests,
> what do they mean?
> If I have failed tests, how do I fix it? If I don't run regression tests, what's
> the consequences I have?
> Would anyone tell me, thanks.
To be honest, the only ppl that should be required to run
regression tests are those that are developing and preparing for
releases...for someone installing, they don't really give a warm fuzzy
feeling due to the discrepencies that the various platforms show that we
consider to be "normal" :(
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
> On Mon, 8 Jun 1998, Doug Lo wrote:
>
> > Hi,
> >
> > I saw the file INSTALL and I ran regression tests, I saw lots of failed tests,
> > what do they mean?
> > If I have failed tests, how do I fix it? If I don't run regression tests, what's
> > the consequences I have?
> > Would anyone tell me, thanks.
>
> To be honest, the only ppl that should be required to run
> regression tests are those that are developing and preparing for
> releases...for someone installing, they don't really give a warm fuzzy
> feeling due to the discrepencies that the various platforms show that we
> consider to be "normal" :(
But INSTALL says:
18) If you wish to skip the regression tests then skip to step 21.
However, we think skipping the tests is a BAD idea!
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
On Mon, 8 Jun 1998, Bruce Momjian wrote:
> > To be honest, the only ppl that should be required to run
> > regression tests are those that are developing and preparing for
> > releases...for someone installing, they don't really give a warm fuzzy
> > feeling due to the discrepencies that the various platforms show that we
> > consider to be "normal" :(
>
> But INSTALL says:
>
> 18) If you wish to skip the regression tests then skip to step 21.
> However, we think skipping the tests is a BAD idea!
and we think this because? its always confused me as to why an
end-user would generally have to run regression tests on "supported and
tested platforms". I can understand us, as developers, doing it prior to
a release, and I can understand someone doing it on an 'untested'
platform...but anything on a supported/tested platform should be caught
by us, the developers, before the end-users see the software...
Now, if we can get the regression tests to pass 100% on all
platforms, the point becomes moot, but, IMHO, all it does is causes/adds
more confusion to the end user then required... :(
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
> > Now, if we can get the regression tests to pass 100% on all > platforms, the point becomes moot, but, IMHO, all it does is causes/adds > more confusion to the end user then required... :( Let's change the INSTALL. We are much more mature now as a product. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
At 7:29 +0300 on 8/6/98, The Hermit Hacker wrote: > and we think this because? its always confused me as to why an > end-user would generally have to run regression tests on "supported and > tested platforms". I can understand us, as developers, doing it prior to > a release, and I can understand someone doing it on an 'untested' > platform...but anything on a supported/tested platform should be caught > by us, the developers, before the end-users see the software... > > Now, if we can get the regression tests to pass 100% on all > platforms, the point becomes moot, but, IMHO, all it does is causes/adds > more confusion to the end user then required... :( May I protest, please? What exactly is a supported/tested platform? Timezone differences make some of the failures, and I think it's important that we recognise them and know that we have a timezone problem. Also, have you really tested the system on all available systems? I saw it compiled for solaris 2.6. Has it been tested for 2.5? Library differences, a slightly different installation procedure, and the regression test points you, at least, in the right direction to ask questions. After all, unix is the administrator's creation, and he/she may decide to move things around. The regression tests tell him if one of his inventions are a bit overboard. End users which merely use the database should not be concerned with such things, but if we are to run the system in a serious environment, my system admin wants to be sure that postgres works *here*. Herouth -- Herouth Maoz, Internet developer. Open University of Israel - Telem project http://telem.openu.ac.il/~herutma
On Mon, 8 Jun 1998, Bruce Momjian wrote:
> >
> > Now, if we can get the regression tests to pass 100% on all
> > platforms, the point becomes moot, but, IMHO, all it does is causes/adds
> > more confusion to the end user then required... :(
>
> Let's change the INSTALL. We are much more mature now as a product.
Agreed, let's just remove the extra line that says that we don't
recommend skipping it...