Thread: pgsql: Improve speed of make check-world
Improve speed of make check-world Before, make check-world would create a new temporary installation for each test suite, which is slow and wasteful. Instead, we now create one test installation that is used by all test suites that are part of a make run. The management of the temporary installation is removed from pg_regress and handled in the makefiles. This allows for better control, and unifies the code with that of test suites not run through pg_regress. review and msvc support by Michael Paquier <michael.paquier@gmail.com> more review by Fabien Coelho <coelho@cri.ensmp.fr> Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/dcae5faccab64776376d354decda0017c648bb53 Modified Files -------------- .gitignore | 1 + GNUmakefile.in | 1 + contrib/earthdistance/Makefile | 2 +- contrib/test_decoding/Makefile | 16 ++- src/Makefile.global.in | 30 +++-- src/bin/pg_upgrade/test.sh | 2 +- src/interfaces/ecpg/test/Makefile | 10 +- src/makefiles/pgxs.mk | 7 +- src/pl/plperl/GNUmakefile | 2 - src/pl/plpython/Makefile | 3 - src/pl/tcl/Makefile | 2 - src/test/isolation/Makefile | 15 +-- src/test/regress/GNUmakefile | 4 - src/test/regress/pg_regress.c | 234 +++++++++--------------------------- src/test/regress/pg_regress.h | 6 - src/test/regress/pg_regress_main.c | 4 +- src/tools/msvc/vcregress.pl | 67 +++++++---- 17 files changed, 145 insertions(+), 261 deletions(-)
On 04/23/2015 10:11 AM, Peter Eisentraut wrote:
> Improve speed of make check-world
>
> Before, make check-world would create a new temporary installation for
> each test suite, which is slow and wasteful. Instead, we now create one
> test installation that is used by all test suites that are part of a
> make run.
>
> The management of the temporary installation is removed from pg_regress
> and handled in the makefiles. This allows for better control, and
> unifies the code with that of test suites not run through pg_regress.
>
> review and msvc support by Michael Paquier <michael.paquier@gmail.com>
>
> more review by Fabien Coelho <coelho@cri.ensmp.fr>
This has caused some buildfarm breakage, in the module that runs the
collate.linux.utf8 test.
Currently it is coded to run like this:
my @checklog;
my $cmd ="./pg_regress --psqldir=$installdir/bin --dlpath=. "
."$inputdir --port=$buildport collate.linux.utf8";
@checklog = `cd $pgsql/src/test/regress && $cmd 2>&1`;
and the error it now gets is:
./pg_regress: unrecognized option '--psqldir=/home/bf/bfr/root/HEAD/inst/bin'
TBH I'm not 100% convinced that removing the ability to have pg_regress
handle temprary installation management is a good thing.
cheers
andrew
Peter Eisentraut wrote: > Improve speed of make check-world > > Before, make check-world would create a new temporary installation for > each test suite, which is slow and wasteful. Instead, we now create one > test installation that is used by all test suites that are part of a > make run. Maybe I'm confused, but now "make -C doc/src/sgml check" seems to build the whole backend; it didn't do so before, and I think that's quite pointless. I take it that's unintended? Can we get the old behavior back? -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, May 5, 2015 at 03:02:50PM -0300, Alvaro Herrera wrote:
> Peter Eisentraut wrote:
> > Improve speed of make check-world
> >
> > Before, make check-world would create a new temporary installation for
> > each test suite, which is slow and wasteful. Instead, we now create one
> > test installation that is used by all test suites that are part of a
> > make run.
>
> Maybe I'm confused, but now "make -C doc/src/sgml check" seems to build
> the whole backend; it didn't do so before, and I think that's quite
> pointless. I take it that's unintended? Can we get the old behavior
> back?
Yes, I found this odd too. It is caused by this commit:
commit dcae5faccab64776376d354decda0017c648bb53
Author: Peter Eisentraut <peter_e@gmx.net>
Date: Thu Apr 23 08:59:52 2015 -0400
Improve speed of make check-world
It basically added a'check' target dependency of 'temp-install' to
Makefile.global.in, which added it to every Makefile containing a
'check' target.
While you can't easily add rule action to an existing target, you can
easily add dependencies by just mentioning the target multiple times,
e.g.
check: dep1
check: dep2
dep1:
echo 1
dep2:
echo 2
Running 'make check' will output '1 2'.
Now, of course many check's need temp-install, but the SGML build does
not. The attached diff uses a symbol defined in doc/src/sgml/Makefile
(GENERATED_SGML) to supress the temp-install target rule action.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
Attachment
On Sat, May 9, 2015 at 02:02:15PM -0400, Bruce Momjian wrote: > Yes, I found this odd too. It is caused by this commit: > > commit dcae5faccab64776376d354decda0017c648bb53 > Author: Peter Eisentraut <peter_e@gmx.net> > Date: Thu Apr 23 08:59:52 2015 -0400 > > Improve speed of make check-world > > It basically added a'check' target dependency of 'temp-install' to > Makefile.global.in, which added it to every Makefile containing a > 'check' target. > > While you can't easily add rule action to an existing target, you can > easily add dependencies by just mentioning the target multiple times, > e.g. > > check: dep1 > > check: dep2 > > dep1: > echo 1 > > dep2: > echo 2 > > > Running 'make check' will output '1 2'. > > Now, of course many check's need temp-install, but the SGML build does > not. The attached diff uses a symbol defined in doc/src/sgml/Makefile > (GENERATED_SGML) to supress the temp-install target rule action. Patch applied, with added Makefile comment. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +
On Tue, May 12, 2015 at 11:01:49AM -0400, Bruce Momjian wrote: > > While you can't easily add rule action to an existing target, you can > > easily add dependencies by just mentioning the target multiple times, > > e.g. > > > > check: dep1 > > > > check: dep2 > > > > dep1: > > echo 1 > > > > dep2: > > echo 2 > > > > > > Running 'make check' will output '1 2'. > > > > Now, of course many check's need temp-install, but the SGML build does > > not. The attached diff uses a symbol defined in doc/src/sgml/Makefile > > (GENERATED_SGML) to supress the temp-install target rule action. > > Patch applied, with added Makefile comment. This patch has broken the build farm, and I am fixing it now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +
On Tue, May 12, 2015 at 12:24:17PM -0400, Bruce Momjian wrote: > > > Running 'make check' will output '1 2'. > > > > > > Now, of course many check's need temp-install, but the SGML build does > > > not. The attached diff uses a symbol defined in doc/src/sgml/Makefile > > > (GENERATED_SGML) to supress the temp-install target rule action. > > > > Patch applied, with added Makefile comment. > > This patch has broken the build farm, and I am fixing it now. Fixed. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +