Thread: Regarding The PGWeb Testing Harness Suite Development in GSOC 2023

On 5/17/23 2:04 PM, Akshat Jaimini wrote:
> Hello everyone!
> 
> We are pleased to announce that the PGWeb Testing Harness Suite for the 
> official website is being developed in GSOC 2023.
> 
> You can checkout the repository for the project here: 
> https://github.com/destrex271/pgweb-testing-harness 
> <https://github.com/destrex271/pgweb-testing-harness> where we'll have 
> all the tests that would be written for the website.

Thanks, this sounds like a great effort.

> It would be amazing if you guys have any suggestions like the features 
> that you would like to have in the project. As of now we are focusing on 
> implementing the functional tests(that includes testing out the links 
> and other interactive elements on the website) using Selenium.

If you want to start with a "challenging" one, I can give a real use 
case where functional tests would have helped. We have a series of pages 
that render information about the release notes[1][2]. There was a 
recent refactor of the code to simplify the implementation of rendering 
a page like [2], but when it was deployed to production, there were 
numerous errors, even after manual testing. Having a thorough set of 
functional tests should have caught most of them, or at least made it 
easier to ensure we were testing for more cases.

I haven't written interactive tests in awhile with Selenium, but back 
when I did, my experience is that it would be too much for the kind of 
interactivity the PostgreSQL website has. The Django test client[4] may 
be good enough for what we need, but I'm happy to be corrected on this.

Thanks,

Jonathan

[1] https://www.postgresql.org/docs/release/
[2] https://www.postgresql.org/docs/release/15.3/
[3] https://github.com/postgres/pgweb/blob/master/pgweb/docs/views.py#L307
[4] https://docs.djangoproject.com/en/4.2/topics/testing/tools/

On 5/17/23 3:40 PM, Akshat Jaimini wrote:
> Thanks a lot for the suggestion. I'll check out if django test client 
> can satisfy our requirements, asap.
> The reason for using selenium earlier was to utilize its capability to 
> conduct cross-browser testing

postgresql.org doesn't have too much functionality that would really 
benefit from cross-browser testing. The CSS layer leverages Bootstrap, 
so we defer cross-browser compatibility to that library. JavaScript is 
pretty minimal, though granted it is used on a critical path, i.e. the 
download page[1].

> as well but if the test client can satisfy 
> the major requirements
> of the functional testing component we'll surely switch over to it for 
> that module. I'll share the results as soon as I am done testing out the 
> test client

That all said, I think we'd get the most impact having tests around the 
other functional units, and I think the Django test client can get us 
pretty far, though if it turns out we'd benefit from Selenium I'd be 
happy to consider it.

Thanks,

Jonathan

[1] https://www.postgresql.org/download/