Re: make installcheck-world in a clean environment - Mailing list pgsql-hackers

Robert Haas <robertmhaas@gmail.com> writes:

After thinking about this some more, I think the question here is
definitional.  A first attempt at defining 'make installcheck' is to
say that it runs the tests from the build tree against the running
server.  Certainly, we intend to use the SQL files that are present in
the build tree, not the ones that were present in the build tree where
the running server was built.  But what about client programs that we
use to connect to the server?  You're suggesting that we use the
pre-installed ones, but that is actually pretty problematic because
the ones we see as installed might correspond neither to the contents
of the build tree nor to the running server.

If the contents of the source tree doesn't correspond to the running server, then I'm afraid, we can't installcheck it for sure.
I think it's supposed that we use for installcheck exactly the same source files that was used to build the server.
Regarding clients program, if we will not use/check it while installchecking, then by what means they can be tested when installed?
I think, the pgsql-packagers would like to check whether the whole installation of PostgreSQL is working, not only the server.
For me, very realistic and most useful scenario of installcheck is:
install postgresql-x.rpm
install postgresql-x.src.rpm
./configure --prefix=$target_path --enable-tap-tests, etc...
make installcheck(-world)

  Conceivably we could end
up having a mix of assets from three different places: (1) the running
server, (2) the build tree, (3) whatever is in our path at the moment.
That seems very confusing.  So now I think it's probably right to
define 'make installcheck' as using the assets from the build tree to
test the running server.  Under that definition, we're missing some
dependencies, but USE_INSTALLED_ASSETS isn't a thing we need.

I see another scenario, that was discussed a month ago - remote (or server-only) installcheck.
( https://www.postgresql.org/message-id/flat/CAM0nTJ6iorX_tmg5MX0mQU3z3w%3D9wk%2BpGK8zrvn7DNWqnyv%2BsQ%40mail.gmail.com )
It can be useful too and if it will be supported, then USE_INSTALLED_ASSETS usage should be extended to psql, etc.

Nah, I disagree with this.  To me, the purpose of "make installcheck"
is to verify the correctness of an installation, which I take to include
the client programs as well as the server.  I think that "make
installcheck" ought to use the installed version of any file that we
actually install, and go to the build tree only for things we don't
install (e.g. SQL test scripts).

Yes, that's the proposed patches intended for.  I didn't encounter any problems with it during internal testing with Linux and mingw-builds.

If the user has screwed up his PATH or other environmental aspects so that
what he's testing isn't a single installation, that's his error, not
something that "make installcheck" ought to work around.  Indeed, maybe
such aspects of his setup are intentional, and second-guessing them would
completely defeat his purpose.  In any case, if you want to test the
build-tree assets, that's what "make check" is for.

Even modified configs could lead to test failures (for example, lc_messages can break server log checking), so the installcheck should be performed against some clean and determinated installation anyway.

Best regards,
------
Alexander Lakhin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company