Re: TAP output format in pg_regress - Mailing list pgsql-hackers

> On 29 Mar 2023, at 09:08, Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote:
>
> On 28.03.23 15:56, Daniel Gustafsson wrote:
>>> On 28 Mar 2023, at 15:26, Daniel Gustafsson <daniel@yesql.se> wrote:
>>> I think the attached is a good candidate for going in, but I would like to see it
>>> for another spin in the CF bot first.
>> Another candidate due to a thinko which raised a compiler warning.
>
> This is incorrect:
>
> -echo "+++ tap install-check in $(subdir) +++" && \
> +echo "\# +++ tap install-check in $(subdir) +++" && \
>
> It actually prints the backslash.
>
> But this appears to be correct:
>
> pg_regress_check = \
> -    echo "+++ regress check in $(subdir) +++" && \
> +    echo "\# +++ regress check in $(subdir) +++" && \
>
> (Maybe because it's in a variable definition?)

Copy paste error, fixed.

> I'm confused why all the messages at the end lost their period ("All tests passed", "A copy of the test summary ...",
etc.). As long as they are written as sentences, they should have a period. 

Fixed.  I also made sure that all messages printing the output directory wrap
it in quotes like how we print paths generally.

> There is a comment "Testnames are indented 8 spaces" but you made that into a macro now, which is currently defined
to3. 

As explained below, this is removed.  I've also removed a comment on NLS which
no longer makes sense.

> There is an unexplained #if 0.

Ugh, I clearly should've stayed on the couch yesterday.

> The additions of
>
> +       free(difffilename);
> +       difffilename = NULL;
> +       free(logfilename);
> +       logfilename = NULL;
>
> in the success case are not clear.  The program is going to end soon anyway.

Fair enough, removed.

> On the indentation of the test names:  If you look at the output of meson test verbose mode (e.g., meson test -C
_build--suite regress --verbose), it reproduces the indentation in a weirdly backwards way, e.g., 
>
> ▶ 1/1 stats                                  981 ms OK
> ▶ 1/1 event_trigger                          122 ms OK
> ▶ 1/1 oidjoins                               172 ms OK
> ▶ 1/1 fast_default                              137 ms OK
> ▶ 1/1 tablespace                                285 ms OK
>
> Not sure by which logic it arrives at that, but you can clearly see 3 additional spaces for the single tests.

I'm not sure what meson does actually.  It seems to strip the leading padding
and line up the testname, but then add the stripped padding on the right side?
Removing the padding for parallel tests solves it, so I have in the attached
moved to indicating parallel tests with a leading '+' and single tests with
'-'.  Not sure if this is clear enough, but it's not worse than padding IMO.

> One thing I have noticed while playing around with this is that it's quite hard to tell casually whether a test run
hasfailed or is failing, if you just keep half an eye on it in another Window.  The display of "ok"/"not ok" as well as
thesummaries at the end are much less prominent than the previous "ok"/"FAILED" and the summary box at the end.  I'm
notsure what to do about that, or if it's just something to get used to. 

meson already presents the results in a box, so if we bring back the === box it
gets quite verbose there.  To some extent I think it is something to get used
to, but if there is discontent with what it looks like reported when more
hackers gets exposed to this we need to fix it.

--
Daniel Gustafsson