I believe Tom / Andres and Fabien all have valid points.
Net-net, I believe the tests although non-negotiable, are not required to be in make-check. For now, its the slow tests that are the pain points here, and then I would soon try to prune them and commit once again.
Whether it goes in make-check or not is obviously not discretion but to me their importance is undoubted since I am sure they increase code-coverage. Actually that is 'how' I create those tests, I look at untouched code and create new tests that check untouched code.
Would try to revert with a faster script (preferably with minimal CREATE/DROP).
On 2013-05-13 16:52:08 +0200, Fabien COELHO wrote: > > Hello, > > >>Would you be okay if there is one/a few effective create/drop (some tests > >>check that the create or drop fails e.g. depending on permissions, which > >>ISTM is not tested anywhere else), so that tests for various ALTER > >>DATABASE commands are combined together onto these databases? > > > >TBH, I do not see that such tests are worth adding, if they are going to > >significantly slow down the core regression tests. Those tests are run > >probably hundreds of times a day, in aggregate across all Postgres > >developers. Adding ten seconds or whatever this would add is a major > >cost, while the benefit appears trivial. > > >We could consider adding expensive low-value tests like these to some > >alternate regression target that's only exercised by buildfarm members, > >perhaps. But I think there's probably a point of diminishing returns > >even in that context. > > I'm not sure that the tests are "low value", because a commit that would > generate a failure on a permission check test would be a potential security > issue for Pg.
> As for the cost, if the proposed tests are indeed too costly, what is not > necessarily the case for what I have seen, I do not think that it would be a > great problem to have two set of tests, with one a superset of the other, > with some convention.
Well, tests like permission tests aren't the expensive part. The actual CREATE/DROP DATABASE you do is. The latter essentially are already tested by the buildfarm already. So, trimming the patch to do only the fast stuff should be less controversial?
Greetings,
Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
