Melanie Plageman <melanieplageman@gmail.com> writes:
> On Sun, Jul 21, 2024 at 5:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I note also that the PG_TEST_EXTRA approach has caused xid_wraparound
>> to get next-to-zero buildfarm coverage. If that test is actually
>> capable of revealing problems, we're unlikely to find out under the
>> status quo.
> What is the argument for PG_TEST_EXTRA if it is not running on almost
> any buildfarm animals? Are some of those tests valuable for other
> reasons than being consistently automatically run (e.g. developer
> understanding of how a particular part of code works)?
The point of PG_TEST_EXTRA is to make some of the tests be opt-in.
Originally it was just used for tests that might have security
implications (e.g. the kerberos tests, which involve running a
not-terribly-locked-down kerberos server). I'm a little suspicious
of using it for tests that merely take an unreasonable amount of
time --- to me, that indicates laziness on the part of the test
author. It'd be better to get the test runtime down to the point
where it's reasonable to expect all the buildfarm animals to run it.
As an example, we're not getting any Valgrind coverage on
xid_wraparound, and we won't ever get it with the current approach,
which I find bad.
> I don't know if you mean that PG_TEST_EXTRA tests are never
> run or just seldom run. If they are never run on the buildfarm, then
> they could end up silently breaking too.
They are opt-in, meaning that both buildfarm owners and regular
developers have to take extra action (i.e. set the PG_TEST_EXTRA
environment variable) to run them. There's a reasonable number
of animals opting into ssl, kerberos, etc, but I see only two
that are opting into xid_wraparound. If we change this new test
to be conditional on PG_TEST_EXTRA, it won't get run unless you
successfully nag some buildfarm owners to run it.
regards, tom lane
