Re: RFC: adding pytest as a supported test framework - Mailing list pgsql-hackers
From | Jelte Fennema-Nio |
---|---|
Subject | Re: RFC: adding pytest as a supported test framework |
Date | |
Msg-id | CAGECzQTEPcLH-Gjr8aTkK=R36_jHU_ikhhan6hO42zCoUNv=yA@mail.gmail.com Whole thread Raw |
In response to | Re: RFC: adding pytest as a supported test framework (Noah Misch <noah@leadboat.com>) |
Responses |
Re: RFC: adding pytest as a supported test framework
Re: RFC: adding pytest as a supported test framework Re: RFC: adding pytest as a supported test framework Re: RFC: adding pytest as a supported test framework |
List | pgsql-hackers |
On Wed, 12 Jun 2024 at 01:48, Noah Misch <noah@leadboat.com> wrote: > If we're going to test in a non-Perl language, I'd pick C over Python. There > would be several other unlikely-community-choice languages I'd pick over > Python (C#, Java, C++). My main goals of this thread are: 1. Allowing people to quickly write tests 2. Have those tests do what the writer intended them to do 3. Have good error reporting by default Those goals indeed don't necesitate python. But I think those are really hard to achieve with any C based framework, and probably with C++ too. Also manual memory management in tests seems to add tons of complexity for basically no benefit. I think C#, Java, Go, Rust, Kotlin, and Swift would be acceptable choices for me (and possibly some more). They allow some type of introspection, they have a garbage collector, and their general tooling is quite good. But I think a dynamically typed scripting language is much more fitting for writing tests like this. I love static typing for production code, but imho it really doesn't have much benefit for tests. As scripting languages go, the ones that are still fairly heavily in use are Javascript, Python, Ruby, and PHP. I think all of those could probably work, but my personal order of preference would be Python, Ruby, Javascript, PHP. Finally, I'm definitely biased towards using Python myself. But I think there's good reasons for that: 1. In the data space, that Postgres in, Python is very heavily used for analysis 2. Everyone coming out of university these days knows it to some extent 3. Multiple people in the community have been writing Postgres related tests in python and have enjoyed doing so (me[1], Jacob[2], Alexander[3]) What language people want to write tests in is obviously very subjective. And obviously we cannot allow every language for writing tests. But I think if ~25% of the community prefers to write their tests in Python. Then that should be enough of a reason to allow them to do so. TO CLARIFY: This thread is not a proposal to replace Perl with Python. It's a proposal to allow people to also write tests in Python. > I also want the initial scope to be the new language coexisting with the > existing Perl tests. If a bulk translation ever happens, it should happen > long after the debut of the new framework. That said, I don't much trust a > human-written bulk language translation to go through without some tests > accidentally ceasing to test what they test in Perl today. I definitely don't think we should rewrite all the tests that we have in Perl today into some other language. But I do think that whatever language we choose, that language should make it as least as easy to write tests, as easy to read them and as easy to see that they are testing the intended thing, as is currently the case for Perl. Rewriting a few Perl tests into the new language, even if not merging the rewrite, is a good way of validating that imho. PS. For PgBouncer I actually hand-rewrote all the tests that we had in bash (which is the worst testing language ever) in Python and doing so actually found more bugs in PgBouncer code that our bash tests wouldn't catch. So it's not necessarily the case that you lose coverage by rewriting tests. [1]: https://github.com/pgbouncer/pgbouncer/tree/master/test [2]: https://github.com/jchampio/pg-pytest-suite [3]: https://github.com/postgrespro/testgres
pgsql-hackers by date: