Re: pg_regress: schedule multi-line test groups - Mailing list pgsql-hackers

From Alexandre Felipe
Subject Re: pg_regress: schedule multi-line test groups
Date
Msg-id CAE8JnxPycpY5mk7C-rNa4jeMQQXG1H5mobB10Oa_CaQs1nBvUg@mail.gmail.com
Whole thread
In response to Re: pg_regress: schedule multi-line test groups  (Daniel Gustafsson <daniel@yesql.se>)
Responses Re: pg_regress: schedule multi-line test groups
List pgsql-hackers

Thank you for your looking into it Daniel

On Fri, Sep 18, 2026 at 10:31 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> On 17 Sep 2026, at 10:43, Alexandre Felipe <o.alexandre.felipe@gmail.com> wrote:

> Actually, I think we should support arbitrary size test groups and run them on a pool
> from a queue that would possibly make the regression faster as today we always
> wait for the slowest test in each group.

I know of ongoing, as of yet unpublished, work in this area to improve this
scheduling which will also change the syntax.  There is a bit complexity to it
than just picking from a pool since there are inter-suite dependencies.

An easy gain would be to somehow transpose execution, currently we put together
things to indicate that they are independent, it would be more readable if we used
the lines to indicate dependency.

The right way to handle dependency is by writing a dependency tree and doing
a topological sort, like makefiles, but to keep the syntax backward compatible 
we could do something like this

# tests that have to run in isolation
test: setup_test
test: sanity_check
# tests that have to respect certain order
test sequentially: create_am psql
# where a test should wait for multiple tests that can run in parallel
test sequentially: (geometry create_index_spgist hash_index brin) amutils
# tests that can mix with other tests
test concurrently: compression compression_lz4 compression_pglz cluster

But please, let's first get the multiline syntax over the fence.

v2 is logging the effective concurrency of the groups during the execution
(we can't optimise what we don't measure).
e.g:
# effective concurrency 5.80 / 18
# effective concurrency 3.83 / 15


        +               /*
        +                * Found `test: # no tests` treat it as a multiline test group
        +                */
                        if (num_tests == 0)
                        {
        -                       bail("syntax error in schedule file \"%s\" line %d: %s",
        -                                schedule, line_num, scbuf);
        +                       multiline_test = true;
        +               }

This will allow empty test groups which we currently treat as an error.  This
schedule:

Fixed, also improved the error messages

Also, if we are to change the accepted syntax, why limit to single line or
multiline, why not accept any whitespace separated name between ^test: and
(^test|EOF)? Something like the below:

        test: boolean
                char
                name
 
Minimalism. in v2 I am trying to do something more like what you described

The schedule syntax in v2 should be

 *   Schedule = (Blank | Comment | Group)*
 *   Group    = "test:" (Line | Comment) (Indent Line)*
 *   Line     = token (Space+ token)* Comment?
 *   Blank    = '\n'
 *   Comment  = '#' [^\n]* \n
 *   Indent   = [\t ]+

But I won't be surprised if you find inconsistencies between that and what
was actually implemented at this stage.


Regards,

Attachment

pgsql-hackers by date:

Previous
From: Tatsuo Ishii
Date:
Subject: Re: Row pattern recognition
Next
From: Alexandre Felipe
Date:
Subject: Re: [patch] Cache invalidation for I/O Workers