Re: tap tests driving the database via psql - Mailing list pgsql-hackers

On 7/27/19 3:15 PM, Andres Freund wrote:
> Hi,
>
> The discussion in [1]
> again reminded me how much I dislike that we currently issue database
> queries in tap tests by forking psql and writing/reading from it's
> stdin/stdout.
>
> That's quite cumbersome to write, and adds a good number of additional
> failure scenarios to worry about. For a lot of tasks you then have to
> reparse psql's output to separate out columns etc.
>
> I think we have a better approach for [1] than using tap tests, but I
> think it's a more general issue preventing us from having more extensive
> test coverage, and especially from having more robust coverage.
>
> I think we seriously ought to consider depending on a proper perl
> database driver.  I can see various approaches:
>
> 1) Just depend on DBD::Pg being installed. It's fairly common, after
>    all. It'd be somewhat annoying that we'd often end up using a
>    different version of libpq than what we're testing against. But in
>    most cases that'd not be particularly problematic.
>
> 2) Depend on DBD::PgPP, a pure perl driver. It'd likely often be more
>    lightweight to install. On the other hand, it's basically
>    unmaintained (last commit 2010), and is a lot less commonly already
>    installed than DBD::Pg. Also depends on DBI, which isn't part of a
>    core perl IIRC.
>
> 3) Vendor a test-only copy of one of those libraries, and build them as
>    part of the test setup. That'd cut down on the number of
>    dependencies.
>
>    But probably not that much, because we'd still depend on DBI, which
>    IIRC isn't part of core perl.
>
>    DBI by default does include C code, and is not that small. There's
>    however a pure perl version maintained as part of DBI, and it looks
>    like it might be reasonably enough sized. If we vendored that, and
>    DBD::PgPP, we'd not have any additional dependencies.
>
>    I suspect that the licensing (dual GPL *version 1* / Artistic
>    License, also V1), makes this too complicated, however.
>
> 4) We develop a fairly minimal pure perl database driver, that doesn't
>    depend on DBI. Include it somewhere as part of the test code, instead
>    of src/interfaces, so it's clearer that it's not ment as an actual
>    official driver.
>
>    The obvious disadvantage is that this would be a noticable amount of
>    code. But it's also not that crazily much.
>
>    One big advantage I can see is that that'd make it a lot easier to
>    write low-level protocol tests. Right now we either don't have them,
>    or they have to go through libpq, which quite sensibly doesn't expose
>    all the details to the outside.  IMO it'd be really nice if we had a
>    way to to write low level protocol tests, especially for testing
>    things like the v2 protocol.
>
> I'm not volunteering to do 4), my perl skills aren't great (if the test
> infra were python, otoh... I have the skeleton of a pure perl driver
> that I used for testing somewhere). But I am leaning towards that being
> the most reasonable choice.
>


+1 for #4.


I'll be happy to participate in any effort.


About 22 years ago I wrote a pure perl implementation of a wire protocol
of roughly similar complexity (RFC1459). I got it basically working in
just a few days, so this sort of thing is very doable. Let's see your
skeleton and maybe it's a good starting point.


cheers


andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services