Re: Adding CI to our tree - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Adding CI to our tree
Date
Msg-id 20220117181946.bmvubqpzqxlvmgeh@alap3.anarazel.de
Whole thread Raw
In response to Re: Adding CI to our tree  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: Adding CI to our tree
Re: Adding CI to our tree
List pgsql-hackers
Hi,

On 2022-01-17 10:25:12 -0500, Andrew Dunstan wrote:
> The buildfarm is moving in the opposite direction, to disaggregate
> steps.

I'm a bit confused as to where you want changes to vcregress.pl
going. Upthread you argued against adding more granular targets to
vcregress. But this seems to be arguing against that?


> There are several reasons for that, including that it makes for
> less log output that you need to churn through o find out what's gone
> wrong in a particular case, and that it makes disabling certain test
> sets via the buildfarm client's `skip-steps' feature possible.

FWIW, to me this shouldn't require a lot of separate manual test
invocations. And continuing to have lots of granular test invocations from the
buildfarm client is *bad*, because it requires constantly syncing up the set
of test targets.

It also makes the buildfarm far slower than necessary, because it runs a lot
of stuff serially that it could run in parallel. This is particularly a
problem for things like valgrind runs, where individual tests are quite slow -
but just throwing more CPUs at it would help a lot.

We should set things up so that:

a) The output of each test can easily be associated with the corresponding set
   of log files.
b) The list of tests run can be determined generically by looking at the
   filesystems
c) For each test run, it's easy to see whether it failed or succeeded

As part of the meson stuff (which has its own test runner), I managed to get a
bit towards this by changing the log output hierarchy so that each test gets
its own directory for log files (regress_log_*, initdb.log, postmaster.log,
regression.diffs, server log files). What's missing is a
test.{failed,succeeded} marker or such, to make it easy to report the log
files for just failed tasks.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: missing indexes in indexlist with partitioned tables
Next
From: Andres Freund
Date:
Subject: Re: slowest tap tests - split or accelerate?