Based on various discussions there seems to be enough desire for writing
tests using pytest. The previous mailinglist discussion[1] became quite
long and discussed various different aspects. So I'm starting a new
thread with my most recent attempt at a "nice to use" framework. It has
many changes since the last one.
The changes resulted from my experience converting ~half of the Perl
tests[2] using an LLM, which revealed gaps and awkward usage patterns.
The last couple of patches in this patchset are a few of those
conversions. These are NOT meant to be committed, at least not until
this thread[3] reaches a conclusion about conversions. They are only
meant to showcase the framework for now.
The patches in the patchset have detailed commit messages. But here's a
short summary per patch:
0001: Small patch to make meson CI output better in general by not
truncating error logs as much
0002: Small meson refactoring, moving a magic number into a variable
0003: Installing 32-bit python, can be removed after pg-vm-images PR is
merged
0004: Basic pytest support for the build system and CI.
0005: Introduction of the actual framework that can manage Postgres
servers and send queries
0006-0007: Two short converted tests that run much faster using this
framework
0008: Shows off the background_sql functionality, which is much easier
to use and more robust than Perl its background_psql
0009: Conversion of the 001_aio test, which is the most unreadable Perl
test I could find (sorry Andres). It becomes, less buggy, more
readable (imo), shorter and faster when written in Python.
0010-0012: Some more tests that are faster and/or easier to read+write
in Python.
[1]:
https://www.postgresql.org/message-id/flat/CAOYmi%2BkThkM9Z87u%3DR_Wi7fCor2i%2BUZKAyq0UCyprzCwTQvqgA%40mail.gmail.com
[2]: https://github.com/JelteF/postgres/pull/36
[3]: https://www.postgresql.org/message-id/flat/cdaaf722-4529-435b-9340-cedf1a3a277f%40dunslane.net
P.S. A full re-review of the framework code is needed, but to the main
things that changed since the first attempt:
1. Each test file now gets its own server, instead of every file sharing
a single one for the whole session.
2. The reloading()/restarting() context managers became plain
append_conf()/adjust_conf() methods.
3. sql() now sends queries using the extended protocol so it can take
query parameters. It also gained batch, oneshot and background variants.
4. pg.sql() reuses a cached connection per test, so the separate conn
fixture is gone.
5. NOTICE/WARNING arrive as Python warnings so they can be asserted on
and so they are logged.
6. PostgresServer grew from 15 to ~55 methods based on missing
functionality. New things that are supported are: backups, streaming,
catchup, logs, injection points, WAL.
7. Installed programs are callables that can be pointed at a server,
e.g. psql("-c", "SELECT 1", server=pg) or
pg_controldata.capture(pg.datadir).
8. PG_TEST_TIMEOUT_DEFAULT now bounds each individual wait or poll
instead of the test as a whole (some tests took longer than the
default for their full runtime.
9. Ports are chosen the same way that Perl does, so the co-operative
locking is shared between the two frameworks to avoid port conflicts.
10. Nearly the whole framework is type-annotated now (155 of 169
function signatures, up from 13 of 139), which editors and pyright
can check.
11. installcheck works now
12. pgxs.mk support
13. Works with the new GitHub Actions CI now.
