Re: [HACKERS] Need a builtin way to run all tests faster manner - Mailing list pgsql-hackers

From Andres Freund
Subject Re: [HACKERS] Need a builtin way to run all tests faster manner
Date
Msg-id 20170311221742.vzqlzzesd5rngtio@alap3.anarazel.de
Whole thread Raw
In response to Re: [HACKERS] Need a builtin way to run all tests faster manner  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: [HACKERS] Need a builtin way to run all tests faster manner  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 2017-03-07 09:36:51 -0300, Alvaro Herrera wrote:
> FWIW, +1 on improving matters here.
>
> Andres Freund wrote:
>
> > The best I can come up so far is a toplevel target that creates the temp
> > install, starts a cluster and then runs the 'installcheck-or-check'
> > target on all the subdirectories via recursion. Individual makefiles can
> > either use the pre-existing cluster (most of of contrib for example), or
> > use the temporary install and run their pre-existing check target using
> > that (the tap tests, test_decoding, ...).
>
> I think a toplevel installcheck-or-check target is a good first step
> (though definitely lets find a better name).  Just being able to run all
> tests without the need for 95% of pointless initdb's would be helpful
> enough.

Here's a hacky proof-of-concept patch that implements a contrib/
'fastcheck' target.
timing of:

make -s -j01 -Otarget check                2m49.286s
make -s -j16 -Otarget check                0m44.120s
make -s -j01 -Otarget fastcheck                1m1.533s
make -s -j16 -Otarget fastcheck                0m24.385s
make -s -j01 -Otarget USE_MODULE_DB=1 \
    installcheck check-test_decoding-recurse    0m56.016s
make -s -j16 -Otarget USE_MODULE_DB=1 \
    installcheck check-test_decoding-recurse    0m23.608s

All of these should, excepting rerunning initdb/server start, have
equivalent test coverage.

'fastcheck' is obviously, if you look at it, a POC. We'd need:
- abstract this away somehow, it's not nice to copy the logic into a
  makefile
- tie contrib/'fastcheck' into check-world by overriding that target
- use the temp install we've created previously?
- more bulletproof server stop mechanism?

- Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachment

pgsql-hackers by date:

Previous
From: Julien Rouhaud
Date:
Subject: [HACKERS] Preserving param location
Next
From: Michael Paquier
Date:
Subject: Re: [HACKERS] scram and \password