Re: [RFC] building postgres with meson - v13 - Mailing list pgsql-hackers

On Sun, Sep 25, 2022 at 5:38 PM Andres Freund <andres@anarazel.de> wrote:
> # run just the main pg_regress tests against existing server
> meson test --setup running main/regress-running

> Peter, would this address your use case?

I tried out your v16 patchset, which seems to mostly work as I'd hoped
it would. Some feedback:

* I gather that "running" as it appears in commands like "meson test
--setup running" refers to a particular setup named "running", that
you invented as part of creating a meson-ish substitute for
installcheck. Can "running" be renamed to something that makes it
obvious that it's a Postgres thing, and not a generic meson thing?

Maybe some kind of consistent naming convention would work best here.
This setup could be "pg_against_running_server", or something along
those lines.

* It would be nice if failed tests told me exactly which "diffs" file
I needed to look at, without my having to look for the message through
the meson log (or running with -v). Is this possible?

To be fair I should probably just be running -v when I run tests
against an existing running server, anyway -- so maybe I'm asking for
the wrong thing. It would at least be slightly better if I always got
to see a path to a .diffs file for failed tests, even without -v. But
it's just a "nice to have" thing -- it's not worth going out of your
way to make it work like that

* Just FYI, there are considerations about the libpq that we link to
here (actually this isn't particularly related to the new installcheck
work, but thought I'd mention it in passing).

I'm using Debian Unstable here. Like Nathan, I found that I needed a
custom -Dextra_lib_dirs just so that binaries would link against the
installation's own libpq, rather than the system libpq. This is
important-ish because linking to the wrong libpq means that I get an
error about not being able to connect via trust authentication to a
unix socket from the directory /var/run/postgresql -- confusion over
where to look for sockets visibly breaks many things.

The workaround that I have is fine, but this still seems like
something that should "just work". I believe that there is a pending
patch for this already, so enough said here.

> I think it'd make sense to add a few toplevel targets to run tests in certain
> ways, but I've not done that here.

I usually run "installcheck-world" (not just installcheck) when I want
to do a generic smoke test with Vaglrind. Sometimes that will fail
relatively early for very silly reasons, for example because I don't
have exactly the expected plan in some harmless way (I try to account
for this by running Valgrind in a shellscript that tries to match
"make check", but that doesn't always work). It is nice that I won't
have to worry about such minor issues derailing everything for a long
running and unsupervised Valgrind test. (Maybe I could have worked
around this before now, but I guess I never tried.)

More generally, I think that we should be encouraging users to think
of the tests as something that you can run in any order. People should
be encouraged to think in terms of the meson abstractions, such as
test setups.

I found that "meson test --setup running --list" will show me what
tests I'll be running if I want to do "installcheck" style testing,
without having to run any tests at all -- another small but important
improvement. This seems worth drawing attention to on the meson Wiki
page as a non-obvious improvement over "installcheck". I might even
find it useful to hard-code some of these tests in a shellscript, that
runs only a subset of "--setup running" tests that happen to be
interesting for Valgrind testing right now.

BTW the meson wiki page iencourages users to think of "meson setup"
and "meson configure" as equivalent to autoconf configure. I get why
you explained it like that, but that confused me at first. What I
since figured out (which will be absurdly obvious to you) is that you
really need to decouple the generic from the specific -- very much
unlike autoconf. I found it useful to separate stuff that I know will
never change for a given build directory (such as the prefix install
path) from other things that are variable configuration settings
(things like the optimization level used by GCC). I now have a
scripted way of running "meson setup" for the former stuff (which is
generic), and a scripted way of running "meson configure" for the
latter set of stuff (with variations for "standard" release and debug
builds, building Valgrind, etc).

-- 
Peter Geoghegan