Re: TAP test breakage on MacOS X - Mailing list pgsql-hackers
From | Andrew Dunstan |
---|---|
Subject | Re: TAP test breakage on MacOS X |
Date | |
Msg-id | 544E6CAA.6020006@dunslane.net Whole thread Raw |
In response to | Re: TAP test breakage on MacOS X (Robert Haas <robertmhaas@gmail.com>) |
Responses |
Re: TAP test breakage on MacOS X
|
List | pgsql-hackers |
On 10/27/2014 11:41 AM, Robert Haas wrote: > On Sun, Oct 26, 2014 at 12:29 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> The larger issue though is that even with both the above things fixed, >> the TAP tests would still be an expensive no-op on the majority of >> buildfarm members. AFAICT, I do not own a single machine on which the >> current TAP tests will consent to run, and in most cases that's after >> going out of my way to fetch CPAN modules that aren't in the vendor Perl >> installs. Peter's mostly been fixing the portability issues by disabling >> tests, which I guess is better than no fix at all, but it leaves darn >> little useful functionality. > I agree, emphatically. Honestly, if we can't get these tests running > everywhere with reasonable effort, we should just rip them out. We've > gone to a lot of trouble in general to make sure that our source code > can be ported even to systems that arguably nobody uses any more, and > instrumental to that effort is keeping the system requirements to > install and test PostgreSQL minimal. At this point, I wouldn't mind > moving the goalposts from C89 to C89 + a bunch of C99 features that > are available on all the platforms we have buildfarm coverage for, and > I wouldn't mind require perl to compile and install, full stop. But > this patch has gone much further than that: you need a new-enough > version of perl, and a new-enough version of a bunch of modules that > aren't installed by default, and maybe not even in the vendor install, > and the documentation on how to make it work is an embarrassment: > > http://www.postgresql.org/docs/devel/static/regress-tap.html > > Beyond all that, I have serious doubts about whether, even if we > eventually get these tests mostly working in most places, whether they > will actually catch any bugs. For example, consider dropdb. The TAP > tests cover the following points: > > - When run with --help or --version, it should exit with code 0, print > something on stderr, and print nothing on stdout. > - When run with --not-a-valid-option, it should exit with a non-zero > exit code, print something on stderr, and print nothing on stdout. > - "dropdb foobar1" should cause "statement: DROP DATABASE foobar1" to > show up in the postgresql log file. > - "dropdb nonexistent" should exit non-zero. > > These are certainly good things to test, but I'd argue that once > you've verified that they are working, they're unlikely to get broken > again in the future. I'm generally supportive of the idea that we > need more automated tests, but these tests seem pretty low-value to > me, even if they were running everywhere, and even moreso if they > aren't. > Yeah. I think at the very least they should be removed from the check-world and installcheck-world targets until this is sorted out. cheers andrew
pgsql-hackers by date: