PG_TEST_EXTRA and meson - Mailing list pgsql-hackers

Hi All,
Using PG_TEST_EXTRA with make is simple, one just sets that environment variable
$ make check
... snip ...

PG_REGRESS='/home/ashutosh/work/units/pghead_make/coderoot/pg/src/test/modules/xid_wraparound/../../../../src/test/regress/pg_regress'
/usr/bin/prove -I ../../../../src/test/perl/ -I .  t/*.pl
# +++ tap check in src/test/modules/xid_wraparound +++
t/001_emergency_vacuum.pl .. skipped: test xid_wraparound not enabled
in PG_TEST_EXTRA
t/002_limits.pl ............ skipped: test xid_wraparound not enabled
in PG_TEST_EXTRA
t/003_wraparounds.pl ....... skipped: test xid_wraparound not enabled
in PG_TEST_EXTRA
Files=3, Tests=0,  1 wallclock secs ( 0.02 usr  0.00 sys +  0.20 cusr
0.03 csys =  0.25 CPU)
Result: NOTESTS

Set PG_TEST_EXTRA
$ PG_TEST_EXTRA=xid_wraparound make check

PG_REGRESS='/home/ashutosh/work/units/pghead_make/coderoot/pg/src/test/modules/xid_wraparound/../../../../src/test/regress/pg_regress'
/usr/bin/prove -I ../../../../src/test/perl/ -I .  t/*.pl
# +++ tap check in src/test/modules/xid_wraparound +++
t/001_emergency_vacuum.pl .. ok
t/002_limits.pl ............ ok
t/003_wraparounds.pl ....... ok
All tests successful.
Files=3, Tests=11, 181 wallclock secs ( 0.03 usr  0.00 sys +  2.87
cusr  3.10 csys =  6.00 CPU)
Result: PASS

But this simple trick does not work with meson
$ meson test  -C $BuildDir --suite setup --suite xid_wraparound
ninja: Entering directory `/home/ashutosh/work/units/pg_review/build_dev'
ninja: no work to do.
1/6 postgresql:setup / tmp_install
    OK                0.85s
2/6 postgresql:setup / install_test_files
    OK                0.06s
3/6 postgresql:setup / initdb_cache
    OK                1.57s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    SKIP              0.24s
5/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    SKIP              0.26s
6/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    SKIP              0.27s

$ PG_TEST_EXTRA=xid_wraparound meson test  -C $BuildDir --suite setup
--suite xid_wraparound
ninja: Entering directory `/home/ashutosh/work/units/pg_review/build_dev'
ninja: no work to do.
1/6 postgresql:setup / tmp_install
    OK                0.41s
2/6 postgresql:setup / install_test_files
    OK                0.06s
3/6 postgresql:setup / initdb_cache
    OK                1.57s
4/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    SKIP              0.20s
5/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    SKIP              0.24s
6/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    SKIP              0.24s

the tests are still skipped.

In order to run these tests, we have to run meson setup again. There
are couple of problems with this
1. It's not clear why the tests were skipped. Also not clear that we
have to run meson setup again - from the output alone
2. Running meson setup again is costly, every time we have to run a
new test from PG_TEST_EXTRA.
3. Always configuring meson with PG_TEST_EXTRA means we will run heavy
tests every time meson test is run. I didn't find a way to not run
these tests as part of meson test once configured this way.

We should either allow make like behaviour or provide a way to not run
these tests even if configured that way.

-- 
Best Wishes,
Ashutosh Bapat