Re: [pgadmin4][patch] Use pytest test runner for unit tests - Mailing list pgadmin-hackers

Known issues:

• Python 2.7, the library we are using for assertions (Grappa) is failing while trying to assert on strings. We created a PR to the library: https://github.com/grappa-py/grappa/pull/43 as soon as this gets in all the tests should pass

Any guesses as to the ETA? Given that most of our dev, and our Windows and Mac packages both run on 2.7 at the moment, it's clear that this is a  required fix before we can proceed.

Attached you can find the patch that bumps grappa to version 0.1.9 that support Python 2.7 

• Jenkins server need a change. Because we now run tests for a single database at a time the Jenkins flow need to change. Our proposal for this is to isolate each database in its own task something similar to the pipeline that we currently use internally:

​

The boxes that we are pointing with the teal arrow represent 1 Python Version Against 1 Database. This can be accomplished using Jenkins as well. In order to accomplish that we are going to generate e Jenkinsfile(https://jenkins.io/doc/book/pipeline/jenkinsfile/) to do this in a more DSL manner, making it easier for us to deploy the pipeline in a more consistent way. The LTS version of jenkins is 2.107.3 the version that we have in CI should be ok, but is always good to be in a LTS version of Jenkins or newer because the community is pretty active.
The idea would be to replace the 5 tasks that we have there and start using a pipeline that will spawn docker containers to isolate the testing between version/database. That is what we do with concourse as we depicted in the previous image.
Does anyone have any thoughts about this?

Well the current public Jenkins system is going away soon, and the new one has had a ton of jobs created that assume the tests run in series automatically against each database server (and is completely different from the current system). I do plan to change that in the longer term, but it requires a change to the way I've been using (and know how to use) Jenkins.

 Do you have a link to the new CI that you can share with us so we can take a pick. We can give some advises on what we believe it is a a good flow for the Continuous Delivery pipeline.

We have some examples from our pipeline that we can share:

Script that we use to run the UT + Feature tests on a docker image that has python+yarn+selenium+postgres installed on it:

https://github.com/greenplum-db/pgadmin4-ci/blob/master/tasks/run-postgres-tests/run.sh

This type of scripts can be added to the Jenkinsfile to create a pipeline step. A good practice in a reproducible pipeline is to use Docker to ensure that every test runs in a clean and predictable environment, this make it easy to reproduce a problem found in testing.

However, the bigger problem for me is that I often run the tests against multiple DB servers on my laptop, without Jenkins. How would that workflow look now?

 

We understand that and it looks like an interesting use case and it could be accomplished by creating a script of something similar that launch the tests multiple times. That is something that we can schedule as a future steps after this patch is reviewed and committed.