Re: can we add subscription TAP test option "vcregress subscriptioncheck" for MSVC builds? - Mailing list pgsql-hackers

On 10/16/21 7:21 AM, Bharath Rupireddy wrote:
> On Wed, Oct 6, 2021 at 4:31 PM Bharath Rupireddy
> <bharath.rupireddyforpostgres@gmail.com> wrote:
>> On Wed, Oct 6, 2021 at 7:52 AM Michael Paquier <michael@paquier.xyz> wrote:
>>> On Wed, Oct 06, 2021 at 07:19:04AM +0530, Bharath Rupireddy wrote:
>>>> I was thinking of the same. +1 for "vcregress check-world" which is
>>>> more in sync with it's peer "make check-world" instead of "vcregress
>>>> all-taptest". I'm not sure whether we can also have "vcregress
>>>> installcheck-world" as well.
>>> check-world, if it spins new instances for each contrib/ test, would
>>> be infinitely slower than installcheck-world.  I recall that's why
>>> Andrew has been doing an installcheck for contribcheck to minimize the
>>> load.  If you run the TAP tests, perhaps you don't really care anyway,
>>> but I think that there is a case for making this set of targets run as
>>> fast as we can, if we can, when TAP is disabled.
>> Out of all the regression tests available with vcregress command
>> today, the tests shown at [1] require an already running postgres
>> instance (much like the installcheck). Should we change these for
>> "vcregress checkworld" so that they spin up tmp instances and run? I
>> don't want to go in this direction and change the code a lot.
>>
>> To be simple, we could just have "vcregress installcheckworld" which
>> requires users to spin up an instance so that the tests shown at [1]
>> would run along with other tests [2] that spins up their own instance.
>> The problem with this approach is that user might setup a different
>> GUC value in the instance that he/she spins up expecting it to be
>> effective for the tests at [2] as well. I'm not sure if anyone would
>> do that. We can ignore "vcregress checkworld" but mention why we don't
>> do this in the documentation "something like it makes tests slower as
>> it spinus up lot of temporary pg instances".
>>
>> Another idea, simplest of all, is that just have "vcregress
>> subscriptioncheck" as proposed in this first mail and not have
>> ""vcregress checkworld" or "vcregress installcheckworld". With this
>> new option and the existing options of vcregress tool, it sort of
>> covers all the tests - core, TAP, contrib, bin, isolation, modules,
>> upgrade, recovery etc.
>>
>> Thoughts?
>>
>> [1]
>> vcregress installcheck
>> vcregress plcheck
>> vcregress contribcheck
>> vcregress modulescheck
>> vcregress isolationcheck
>>
>> [2]
>> vcregress check
>> vcregress ecpgcheck
>> vcregress bincheck
>> vcregress recoverycheck
>> vcregress upgradecheck
>> vcregress subscriptioncheck
> The problems with having "vcregress checkworld" are: 1) required code
> modifications are more as the available "vcregress" test functions,
> which required pre-started pg instance, can't be used directly. 2) it
> looks like spinning up a separate postgres instance for all module
> tests takes time on Windows which might make the test time longer. If
> we were to have "vcregress installcheckworld", we might have to add
> new code for converting some of the existing functions to not use a
> pre-started pg instance.
>
> IMHO, we can just have "vcregress subscriptioncheck" and let users
> decide which tests they want to run.
>
> I would like to hear more opinions on this.
>

My opinion hasn't changed. There is already a way to spell this and I'm
opposed to adding more such specific tests to vcregress.pl. Every such
addition we make increases the maintenance burden.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com