On 11.08.22 06:04, Andres Freund wrote:
> At the moment there's three suites differentiating by the type of test:
> 'pg_regress', 'isolation' and 'tap'. There's also a separate "axis" of suites,
> describing what's being tested, e.g. 'main', 'test_decoding', 'recovery' etc.
>
> That currently works out to each test having two suites, although I've
> wondered about adding a 'contrib' suite as well.
I'm not sure what the value of these suite names would be. I don't
usually find myself wanting to run, say, just all tap tests.
Perhaps suites would be useful to do things like select slow tests
(numeric_big) or handle the ssl, ldap, kerberos tests.
> = Log and Data locations =
>
> To make things like the selection of log files for a specific test easier,
> I've so far set it up so that test data and logs are stored in a separate
> directory from the sources.
>
> testrun/<main|recovery|...>/<testname>/<log|tmp_check|results...>
>
> The runner now creates a test.start at the start of a test and either
> test.success or test.failure at the end. That should make it pretty easy for
> e.g. the buildfarm and CI to make the logs for a failed test easily
> accessible. I've spent far too much time going through the ~hundred logs in
> src/test/recovery/ that the buildfarm displays as one thing.
I don't really understand which problem this solves and how. Sure, the
test output is somewhat complex, but I know where it is and I've never
found myself wishing it to be somewhere else.
