On 18 March 2012 16:13, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Is there a really strong reason why adequate regression testing isn't
> possible in a plain-vanilla pg_regress script? A quick look at the
> script says that it's just doing some SQL commands and then checking the
> results of queries on the pg_stat_statements views. Admittedly the
> output would be bulkier in pg_regress, which would mean that we'd not
> likely want several hundred test cases. But IMO the objective of a
> regression test is not to memorialize every single case the code author
> thought about during development. ISTM it would not take very many
> cases to have reasonable code coverage.
Hmm. It's difficult to have much confidence that a greatly reduced
number of test cases ought to provide sufficient coverage. I don't
disagree with your contention, I just don't know how to judge this
matter. Given that there isn't really a maintenance burden with
regression tests, I imagine that that makes it compelling to be much
more inclusive.
The fact that we rely on there being no concurrent queries might have
to be worked around for parallel scheduled regression tests, such as
by doing everything using a separate database, with that database oid
always in the predicate of the query checking the pg_stat_statements
view.
I probably would have written the tests in Perl in the first place,
but I don't know Perl. These tests existed in some form from day 1, as
I followed a test-driven development methodology, and needed to use a
language that I could be productive in immediately. There is probably
no reason why they cannot be re-written in Perl, but spit out
pg_regress tests, compacting the otherwise-verbose pg_regress input.
Should I cut my teeth on Perl by writing the tests to do so? How might
this be integrated with the standard regression tests, if that's
something that is important?
--
Peter Geoghegan http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services
