Re: TAP output format in pg_regress - Mailing list pgsql-hackers

From Andres Freund
Subject Re: TAP output format in pg_regress
Date
Msg-id 20220704200617.vmuu3cuhhfmnzzpm@awork3.anarazel.de
Whole thread Raw
In response to Re: TAP output format in pg_regress  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: TAP output format in pg_regress
List pgsql-hackers
Hi,

> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> > I'm not sure what to make of all these options.  I think providing a TAP
> > output for pg_regress is a good idea.  But then do we still need the old
> > output?  Is it worth maintaining two output formats that display exactly
> > the same thing in slightly different ways?
>
> Probably is, because this is bad:

> > ... The proposed default format now hides the
> > fact that some tests are started in parallel.  I remember the last time
> > I wanted to tweak the output of the parallel tests, people were very
> > attached to the particular timing and spacing of the current output.  So
> > I'm not sure people will like this.
>
> and so is this:
>
> > The timing output is very popular.  Where is that in the TAP output?
>
> Both of those things are fairly critical for test development.  You
> need to know what else might be running in parallel with a test case,
> and you need to know whether you just bloated the runtime unreasonably.

That should be doable with tap as well - afaics the output of that could
nearly be the same as now, preceded by a #.

The test timing output could (and I think should) also be output - but if I
read the tap specification correctly, we'd either need to make it part of the
test "description" or on a separate line.


On 2022-07-04 10:39:37 -0400, Tom Lane wrote:
> More generally, I'm unhappy about the proposal that TAP should become
> the default output.  There is nothing particularly human-friendly
> about it, whereas the existing format is something we have tuned to
> our liking over literally decades.  I don't mind if there's a way to
> get TAP when you're actually intending to feed it into a TAP-parsing
> tool, but I am not a TAP-parsing tool and I don't see why I should
> have to put up with it.

I'm mostly interested in the tap format because meson's testrunner can parse
it - unsurprisingly it doesn't understand the current regress output. It's a
lot nicer to immediately be pointed to the failed test(s) than having to scan
through the output "manually".

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: pgsql: Change timeline field of IDENTIFY_SYSTEM to int8
Next
From: Andres Freund
Date:
Subject: Re: TAP output format in pg_regress