Re: Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS? - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS?
Date
Msg-id 1060267.1620827410@sss.pgh.pa.us
Whole thread Raw
In response to Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS?  (David Rowley <dgrowleyml@gmail.com>)
Responses Re: Do we need to rethink how to parallelize regression tests to speedup CLOBBER_CACHE_ALWAYS?
List pgsql-hackers
David Rowley <dgrowleyml@gmail.com> writes:
> Right now we start 1 backend for each test in a parallel group then
> wait for the final backend to complete before running the next group.

> Is a particular reason for it to work that way?

There are a whole lot of cases where test Y depends on an earlier test X.
Some of those dependencies are annotated in parallel_schedule, but I fear
most are not.

If we had a full list of such dependencies then we could imagine building
a job scheduler that would dispatch any script that has no remaining
dependencies.

The cases where "script X can't run concurrently with script Y" are
also problematic.  It's not as easy to discover those through testing,
since it might happen to work depending on timing.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Amit Langote
Date:
Subject: Re: Feedback on table expansion hook (including patch)
Next
From: Julien Rouhaud
Date:
Subject: Re: RFC: Logging plan of the running query