Re: tap tests driving the database via psql - Mailing list pgsql-hackers
| From | Craig Ringer |
|---|---|
| Subject | Re: tap tests driving the database via psql |
| Date | |
| Msg-id | CAMsr+YGVLpNa5rkfyAxnSqJBF=Ja_n=CsBL2MCCCNTWMO_1xQQ@mail.gmail.com Whole thread Raw |
| In response to | Re: tap tests driving the database via psql (Andres Freund <andres@anarazel.de>) |
| List | pgsql-hackers |
On Wed, 31 Jul 2019 at 10:20, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2019-07-31 09:32:10 +0800, Craig Ringer wrote:
> OK. So rather than building our own $everything from scratch, lets look at
> solving that.
IDK, a minimal driver that just does what we need it to do is a few
hundred lines, not more. And there's plenty of stuff that we simply
won't be able to test with any driver that's not purposefully written
for testing. There's e.g. basically no way with any of the drivers to
test intentionally bogus sequences of protocol messages, yet that's
something we really ought to test.
I've written custom hacks^Wprograms to tests things like that a few
times. That really shouldn't be necessary.
That's a good point. I've had to write simple protocol hacks myself, and having a reusable base tool for it would indeed be valuable.
I'm just worried it'll snowball into Yet Another Driver We Don't Need, or land up as a half-finished poorly-maintained burden nobody wants to touch. Though the general fondness for and familiarity with Perl in the core circle of Pg contributors and committers would probably help here, since it'd inevitably land up being written in Perl...
I'm interested in evolution of the protocol and ways to enhance it, and I can see something we can actively use for protocol testing being valuable. If done right, the protocol implementation could probably pass-through inspect messages from regular libpq etc as well as serving as either a client or even as a dumb server simulator for pregenerated responses for client testing.
We certainly couldn't do anything like that with existing tools and by reusing existing drivers.
I wonder if there's any way to make writing and maintaining it less painful though. Having to make everything work with Perl 5.8.8 and no non-core modules leads to a whole pile of bespoke code and reinvention.
> Perl modules support local installations. Would it be reasonable to require
> DBI to be present, then rebuild DBD::Pg against our libpq during our own
> test infra compilation, and run it with a suitable LD_LIBRARY_PATH or using
> rpath? That'd actually get us a side-benefit of some test coverage of
> DBD::Pg and libpq.
You're intending to download DBD::Pg?
That'd be a reasonable option IMO, yes. Either via git with https, or a tarball where we check a signature. So long as it can use any pre-existing local copy without needing to hit the Internet, and the build can also succeed without the presence of it at all, I think that'd be reasonable enough.
But I'm probably getting contaminated by excessive exposure to Apache Maven when I have to work with Java. I'm rapidly getting used to downloading things being a part of builds. Personally I'd expect that unless specifically testing new libpq functionality etc, most people would just be using their system DBD::Pg... and I consider linking the system DBD::Pg against our new-compiled libpq more feature than bug, as if we retain the same soname we should be doing a reasonable job of not breaking upward compatibility anyway. (It'd potentially be an issue if you have a very new system libpq and are running tests on an old postgres, I guess).
Or how would you get around the licensing issues? Downloading it will have some issues too: For one at least I
often want to be able to run tests when offline; there's security
concerns too.
Roughly what I was thinking of was:
For Pg source builds (git, or dist tarball), we'd generally curl a tarball of DBD::Pg from a HTTPs URL specified in Makefile variables (with =? so it can be overridden to point to an alt location, different version, local path, etc) and unpack it, then build it. The makefile can also store a signing key fingerprint so we can download the sig file and check the sig is by the expected signer if the user has imported the signing key for DBD::Pg.
We could test if the target directory already exists and is populated and re-use it, so people can git-clone DBD::Pg if they prefer.
And we'd allow users to specify --with-system-dbd-pg at configure time, or --without-dbd-pg .
The Pg perl libraries for our TAP test harness/wrappers would offer a simple function to skip a test if DBD::Pg is missing, a simple function to skip a test if the loaded DBD::Pg lacks $some_feature_or_attribute, etc.
Greetings,
Andres Freund
pgsql-hackers by date: