On Wed, Sep 4, 2024 at 9:17 AM Peter Smith <smithpb2250@gmail.com> wrote:
>
> On Tue, Sep 3, 2024 at 9:23 PM Zhijie Hou (Fujitsu)
> <houzj.fnst@fujitsu.com> wrote:
> >
> > I confirmed that it only increased the testing time by 1 second on my machine.
> >
>
> It seems a pity to throw away perfectly good test cases just because
> they increase how long the suite takes to run.
>
We can take every possible test, but I worry about the time they
consume without adding much value and the maintenance burden. I feel
like core-code we should pay attention to tests as well and don't try
to jam all the possible tests testing mostly similar stuff. Each time
before committing or otherwise verifying the patch, we run make
check-world, so don't want that time to go enormously high. Having
said that, I don't want the added functionality shouldn't be tested
properly and I try my best to achieve that.
> This seems like yet another example of where we could have made good
> use of the 'PG_TEST_EXTRA' environment variable. I have been trying to
> propose adding "subscription" support for this in another thread [1].
> By using this variable to make some tests conditional, we could have
> the best of both worlds. e.g.
> - retain all tests, but
> - by default, only run a subset of those tests (to keep default test
> execution time low).
>
> I hope that if the idea to use PG_TEST_EXTRA for "subscription" tests
> gets accepted then later we can revisit this, and put all the removed
> extra test cases back in again.
>
I am not convinced that tests that are less useful than others or are
increasing the time are good to be kept under PG_TEST_EXTRA but if
more people advocate such an approach then it is worth considering.
--
With Regards,
Amit Kapila.
