Thread: Re: Rewriting the test of pg_upgrade as a TAP test
On 2017-04-03 11:22:02 -0400, Peter Eisentraut wrote: > On 4/3/17 09:07, Michael Paquier wrote: > > I had for some time a WIP patch on which dust has accumulated, so > > attached is a more polished version. In more details, here is what > > happens: > > - test.sh is removed. > > - vcregress.pl loses upgradecheck. > > - The new test is added. In the case of MSVC this is now part of bincheck. > > Patch has been tested on macos and Windows. > > This is a useful start. What I'd really like to see is that instead of > running the full serial tests to populate the pre-upgrade database, we > determine a useful subset of what that ends up generating and just > populate with that. That doesn't strike as particularly future proof. We intentionally leave objects behind pg_regress runs, but that only works if we actually run them... Greetings, Andres Freund
On 4/3/17 11:32, Andres Freund wrote: > That doesn't strike as particularly future proof. We intentionally > leave objects behind pg_regress runs, but that only works if we actually > run them... I generally agree with the sentiments expressed later in this thread. But just to clarify what I meant here: We don't need to run a, say, 1-minute serial test to load a few "left behind" objects for the pg_upgrade test, if we can load the same set of objects using dedicated scripting in say 2 seconds. This would make both the pg_upgrade tests faster and would reduce the hidden dependencies in the main tests about which kinds of objects need to be left behind. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Apr 4, 2017 at 10:52 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote: > On 4/3/17 11:32, Andres Freund wrote: >> That doesn't strike as particularly future proof. We intentionally >> leave objects behind pg_regress runs, but that only works if we actually >> run them... > > I generally agree with the sentiments expressed later in this thread. > But just to clarify what I meant here: We don't need to run a, say, > 1-minute serial test to load a few "left behind" objects for the > pg_upgrade test, if we can load the same set of objects using dedicated > scripting in say 2 seconds. This would make both the pg_upgrade tests > faster and would reduce the hidden dependencies in the main tests about > which kinds of objects need to be left behind. Making the tests run shorter while maintaining the current code coverage is nice. But this makes more complicated the test suite maintenance as this needs either a dedicated regression schedule or an extra test suite where objects are created just for the sake of pg_upgrade. This increases the risks of getting a rotten test suite with the time if patch makers and reviewers are not careful. -- Michael