Re: RFC: adding pytest as a supported test framework - Mailing list pgsql-hackers

On Wed, Dec 17, 2025 at 8:10 AM Andres Freund <andres@anarazel.de> wrote:
> I assume this intentionally doesn't pass CI:
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F6045

v2 intentionally doesn't pass CI to show what the failure modes are.
Jelte rightly pointed out that this masks known 32-bit failures -- I
hadn't yet found a way to get 32-bit and 64-bit Python to behave well
together on Debian. (Not a huge fan of how v4 is working around the
problem, though; if we can't load libpq into Python then why run
pytest?)

Before it gets too far away from me: note that I have not yet been
able to get up to speed with the combined refactoring+feature patch
that Jelte added in v3, and it's now up to v7, and this patchset has
been moved out of Drafts. That's fine by me, but I plan to focus on
things that need to get into PG19 before I focus on this, since
nothing is really blocked on it.

> I agree with adding tap to the configuration summary, but I don't understand
> the prove part, that just seems like a waste of vertical space.

I don't have a strong opinion, if no one finds it useful.

> Yes, that needs to be baked into the image. Chocolatey is catastrophically
> slow and unreliable. It's also just bad form to hit any service with such
> repeated downloads.

Yep.

> Why do we need pytest the program at all?  Running the tests one-by-one with
> pytest as a runner doesn't seem to make a whole lot of sense to me.

Even if we find reasons to implement our own custom runner for mtest
-- which I don't think we should do for the sake of architectural
purity alone; it doesn't seem to be a big deal in practice -- using
pytest directly ensures that the CI is using the same code paths as
individual developers who choose to use pytest as the top-level
runner.

> > -SUBDIRS = perl postmaster regress isolation modules authentication recovery subscription
> > +SUBDIRS = \
> > +     authentication \
> > +     isolation \
> > +     modules \
> > +     perl \
> > +     postmaster \
> > +     pytest \
> > +     recovery \
> > +     regress \
> > +     subscription
>
> I'm onboard with that, but we should do it separately and probably check for
> other cases where we should do it at the same time.

I'm not sure what context this is referring to? What are you on board with?

> I think it'd be a seriously bad idea to start with no central infrastructure,
> we'd be force to duplicate that all over.

Right, I just want central infra to be pulled out of the new tests
that need them rather than the other way around.

> Eventually we'll be forced to
> introduce some central infrastructure, but we'll probably not go around and
> carefully go through the existing tests for stuff that should now use the
> common infrastructure.

Ehh... I don't want to encourage "regular" refactoring of test
fixtures, since that would be incredibly disruptive and cause backport
difficulties, but I think it's fine to expect some careful suite-wide
improvements to be made as we introduce a completely new way of
testing. (And I find composed tests in Python a lot easier to refactor
than Test::More scripts, since they've declared their logical
dependencies.)

Thanks!
--Jacob