Re: tap tests driving the database via psql - Mailing list pgsql-hackers

On Sat, Jul 27, 2019 at 12:15:23PM -0700, Andres Freund wrote:
> Hi,
> 
> The discussion in [1]
> again reminded me how much I dislike that we currently issue database
> queries in tap tests by forking psql and writing/reading from it's
> stdin/stdout.
> 
> That's quite cumbersome to write, and adds a good number of additional
> failure scenarios to worry about. For a lot of tasks you then have to
> reparse psql's output to separate out columns etc.
> 
> I think we have a better approach for [1] than using tap tests, but I
> think it's a more general issue preventing us from having more extensive
> test coverage, and especially from having more robust coverage.
> 
> I think we seriously ought to consider depending on a proper perl
> database driver.  I can see various approaches:
> 
> 1) Just depend on DBD::Pg being installed. It's fairly common, after
>    all. It'd be somewhat annoying that we'd often end up using a
>    different version of libpq than what we're testing against. But in
>    most cases that'd not be particularly problematic.
> 
> 2) Depend on DBD::PgPP, a pure perl driver. It'd likely often be more
>    lightweight to install. On the other hand, it's basically
>    unmaintained (last commit 2010), and is a lot less commonly already
>    installed than DBD::Pg. Also depends on DBI, which isn't part of a
>    core perl IIRC.
> 
> 3) Vendor a test-only copy of one of those libraries, and build them as
>    part of the test setup. That'd cut down on the number of
>    dependencies.
> 
>    But probably not that much, because we'd still depend on DBI, which
>    IIRC isn't part of core perl.
> 
>    DBI by default does include C code, and is not that small. There's
>    however a pure perl version maintained as part of DBI, and it looks
>    like it might be reasonably enough sized. If we vendored that, and
>    DBD::PgPP, we'd not have any additional dependencies.
> 
>    I suspect that the licensing (dual GPL *version 1* / Artistic
>    License, also V1), makes this too complicated, however.
> 
> 4) We develop a fairly minimal pure perl database driver, that doesn't
>    depend on DBI. Include it somewhere as part of the test code, instead
>    of src/interfaces, so it's clearer that it's not ment as an actual
>    official driver.

There's one that may or may not need updates that's basically just a
wrapper around libpq.

https://ftp.postgresql.org/pub/projects/gborg/pgperl/stable/

>    The obvious disadvantage is that this would be a noticable amount of
>    code. But it's also not that crazily much.
> 
>    One big advantage I can see is that that'd make it a lot easier to
>    write low-level protocol tests. Right now we either don't have them,
>    or they have to go through libpq, which quite sensibly doesn't expose
>    all the details to the outside.  IMO it'd be really nice if we had a
>    way to to write low level protocol tests, especially for testing
>    things like the v2 protocol.

That sounds worth doing as a separate thing, and an obvious
application of it would be something like a febesmith, which would get
us a better idea as to whether we've implemented the protocol we say
we have.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate