Hi,
On 2022-09-25 12:38:06 -0700, Andres Freund wrote:
> On 2022-09-24 17:33:49 -0700, Peter Geoghegan wrote:
> > On Sat, Sep 24, 2022 at 5:13 PM Andres Freund <andres@anarazel.de> wrote:
> > > > One more question about this, that wasn't covered by the Wiki page: is
> > > > there some equivalent to "make installcheck" with meson builds?
> > >
> > > Not yet. Nothing impossible, just not done yet. Partially because installcheck
> > > is so poorly defined (run against an already running server for pg_regress vs
> > > using "system" installed binaries for tap tests).
> >
> > Got it. I can work around that by just having an old autoconf-based
> > vpath build directory. I'll need to do this when I run Valgrind.
> >
> > My workaround would be annoying if I needed to run "installcheck"
> > anywhere near as frequently as I run "make check-world". But that
> > isn't the case. meson delivers a significant improvement in the metric
> > that really matters to me, so I can't really complain.
>
> My gut feeling is that we should use this opportunity to split 'installcheck'
> into two. "test a running server" and "test installed binaries".
>
> I think the cleanest way to do this with meson would be to utilize meson
> tests's "setups".
> $ meson test --setup 'running-server'
> would run all [selected] tests compatible with running against a running
> server. And
> $ meson test --setup 'installed'
> would test installed binaries.
I've added support for a 'running' setup in the attached rebased series. A
bunch of preparatory changes were necessary - as it turns out we've introduced
a bunch of role name conflicts between tests.
I had to set it up so that the main regress and isolationtester tests don't
run in parallel with other tests, because they don't pass reliably due to
checking pg_locks etc.
I also found a problem independent of meson [1] / installcheck.
# run all tests that support running against existing server
meson test --setup running
# run just the main pg_regress tests against existing server
meson test --setup running main/regress-running
I've also worked some more on cleaning up other patches in the series,
particularly the precompiled headers one (interesting because it reduces
windows compile times noticably).
Peter, would this address your use case?
I think it'd make sense to add a few toplevel targets to run tests in certain
ways, but I've not done that here.
Greetings,
Andres Freund
[1] https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2%40awork3.anarazel.de
