Postgres Pro
Downloads
Home   >   mailing lists

Re: pgAdmin4: Test result enhancement patch - Mailing list pgadmin-hackers

From Navnath Gadakh
Subject Re: pgAdmin4: Test result enhancement patch
Date
Msg-id CAOAJCYp-rJHKqJjjkxZQnhk2Ed+ap_b_rdM-TPoOgt7kh8eVhQ@mail.gmail.com
Whole thread Raw
In response to Re: pgAdmin4: Test result enhancement patch  (Dave Page <dpage@pgadmin.org>)
Responses Re: pgAdmin4: Test result enhancement patch  (Dave Page <dpage@pgadmin.org>)
List pgadmin-hackers
Tree view
Hello Dave,

On Fri, Mar 24, 2017 at 9:10 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

On Fri, Mar 24, 2017 at 3:13 PM, Navnath Gadakh
<navnath.gadakh@enterprisedb.com> wrote:
>
>> When running with the patch:
>>
>> 1) The browser isn't closed, and the script never exits - it just sits
>> indefinitely at:
>>
>> =====
>> Please check output in file:
>> /Users/dpage/git/pgadmin4/web/regression/regression.log
>>
>> make: *** [check] Error 1
>> =====
>>
>> without returning to a shell prompt. The browser exits when I hit Ctrl+C.

The above is still a problem. In fact, not only do I have to hit
Ctrl+C, but then the browser prompts me to check I really do want to
exit.

There's also another problem that just showed up. I got the following
failure on PG 9.4 (due to a known intermittent bug that Ashesh and
Tira@Pivotal are working on). Note how it's not reported in the
summary (or the JSON output):

I found the issue, In the feature tests we need to add a scenario name for each test case. the purpose of this patch is to print the failed/skipped test class name with the scenario name like:

152 tests passed

1 test failed:

         LoginRoleGetTestCase (Check Role Node)

16 tests skipped:

SynonymGetTestCase (Fetch synonym Node URL)

But our in-built test framework does not provide that scenario name with failed/skipped test case that's why I override apply_scenario() function.

def apply_scenario(scenario, test):

       name, parameters = scenario

      parameters["scenario_name"] = name

While printing the result, I have checked the if 'scenario_name' in test as we need to print scenario name in test summary as well as in JSON file.

I can do it without scenario name but for better understanding which test scenario is failed it's good to add a scenario name with each test case.

See this is how test cases looks like while printing on console

 API:

runTest (pgadmin.browser.server_groups.servers.databases.schemas.types.tests.test_types_put.TypesUpdateTestCase)

Update type under schema node ... ok

 Feature tests:

runTest (pgadmin.utils.tests.test_versioned_template_loader.TestVersionedTemplateLoader)
... ok

No scenario name in feature tests.


 but our inbuilt test freamework does not provide 

runTest (pgadmin.feature_tests.connect_to_server_feature_test.ConnectsToServerFeatureTest)
... ERROR
runTest (pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)
... ok
runTest (pgadmin.utils.tests.test_versioned_template_loader.TestVersionedTemplateLoader)
... ok

======================================================================
ERROR: runTest (pgadmin.feature_tests.connect_to_server_feature_test.ConnectsToServerFeatureTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/connect_to_server_feature_test.py",
line 37, in runTest
    self._tables_node_expandable()
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/connect_to_server_feature_test.py",
line 73, in _tables_node_expandable
    self.page.toggle_open_tree_item('test_table')
  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 68, in toggle_open_tree_item
    self.find_by_xpath("//*[@id='tree']//*[.='" + tree_item_text +
"']/../*[@class='aciTreeButton']").click()
  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 71, in find_by_xpath
    return self.wait_for_element(lambda driver:
driver.find_element_by_xpath(xpath))
  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 128, in wait_for_element
    return self._wait_for("element to exist", element_if_it_exists)
  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 162, in _wait_for
    "Timed out waiting for " + waiting_for_message)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py",
line 80, in until
    raise TimeoutException(message, screen, stacktrace)
TimeoutException: Message: Timed out waiting for element to exist


----------------------------------------------------------------------
Ran 153 tests in 60.698s

FAILED (errors=1, skipped=12)

...
...
...
This error will be in shown in the summary as well as in JSON file when we add 'scenario_name' for each feature test. 

If you have any other thinking please let me know.

======================================================================
Test Result Summary
======================================================================

Regression - EPAS 9.5:

153 tests passed
0 tests failed
0 tests skipped

Regression - PG 9.5:

141 tests passed
0 tests failed
12 tests skipped:
SynonymGetTestCase (Fetch synonym Node URL)
PackageDeleteTestCase (Fetch Package Node URL)
ResourceGroupsGetTestCase (Get resource groups)
SynonymDeleteTestCase (Fetch synonym Node URL)
ResourceGroupsAddTestCase (Add resource groups)
PackagePutTestCase (Fetch Package Node URL)
SynonymPutTestCase (Fetch synonym Node URL)
ResourceGroupsPutTestCase (Put resource groups)
ResourceGroupsDeleteTestCase (Delete resource groups)
SynonymAddTestCase (Default Node URL)
PackageAddTestCase (Fetch Package Node URL)
PackageGetTestCase (Fetch Package Node URL)

Regression - PG 9.4:

141 tests passed
0 tests failed
12 tests skipped:
SynonymGetTestCase (Fetch synonym Node URL)
PackageDeleteTestCase (Fetch Package Node URL)
ResourceGroupsGetTestCase (Get resource groups)
SynonymDeleteTestCase (Fetch synonym Node URL)
ResourceGroupsAddTestCase (Add resource groups)
PackagePutTestCase (Fetch Package Node URL)
SynonymPutTestCase (Fetch synonym Node URL)
ResourceGroupsPutTestCase (Put resource groups)
ResourceGroupsDeleteTestCase (Delete resource groups)
SynonymAddTestCase (Default Node URL)
PackageAddTestCase (Fetch Package Node URL)
PackageGetTestCase (Fetch Package Node URL)

======================================================================
Thanks. 




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



--
Regards,
Navnath Gadakh

EnterpriseDB Corporation
The Enterprise PostgreSQL Company


pgadmin-hackers by date:

Previous
From: Dave Page
Date:
Subject: pgAdmin 4 commit: Don't require an auth key for the static helpfiles.
Next
From: Akshay Joshi
Date:
Subject: Re: Patch for Feature #1344 Can't use multiplemonitors (Windows 10)