Re: [pgAdmin4][Patch] Feature #3270 Add support for running regression tests against Firefox - Mailing list pgadmin-hackers
Any progress on this Akshay?
On Tue, Apr 24, 2018 at 3:34 PM, Joao De Almeida Pereira <jdealmeidapereira@pivotal.io> wrote:
Hi Akshay,Linter fails and tests are failing when we run the patch on our machine using Firefox. The linter output is:./regression/runtests.py:184: [E125] continuation line with same indent as next logical line./regression/runtests.py:184: [E501] line too long (80 > 79 characters)1 E125 continuation line with same indent as next logical line1 E501 line too long (80 > 79 characters)2The tests output is:=============Running the test cases for 'PostgreSQL 10'============= runTest (pgadmin.feature_tests.copy_selected_query_results_ feature_test. CopySelectedQueryResultsFeatur eTest) Copy rows, column using button and keyboard shortcut ... ERROR runTest (pgadmin.feature_tests. keyboard_shortcut_test. KeyboardShortcutFeatureTest) Test for keyboard shortcut ... Executing shortcut: File main menu...OK Executing shortcut: Object main menu...OK ok runTest (pgadmin.feature_tests.pg_ datatype_validation_test. PGDataypeFeatureTest) Test checks for PG data-types output ... ERROR runTest (pgadmin.feature_tests.query_ tool_journey_test. QueryToolJourneyTest) Tests the path through the query tool ... ERROR runTest (pgadmin.feature_tests.query_ tool_tests. QueryToolFeatureTest) Query tool feature test ... On demand query result... On demand result set on scrolling... OK. On demand result set on grid select all... OK. On demand result set on column select all... OK. Explain query with verbose and cost... ERROR runTest (pgadmin.feature_tests.table_ ddl_feature_test. TableDdlFeatureTest) Test table DDL generation ... ok runTest (pgadmin.feature_tests.view_ data_dml_queries. CheckForViewDataTest) Validate Insert, Update operations in View/Edit data with given test data ... ERROR ERROR runTest (pgadmin.feature_tests.xss_ checks_file_manager_test. CheckFileManagerFeatureTest) Tests to check if File manager is vulnerable to XSS ... ERROR runTest (pgadmin.feature_tests.xss_ checks_panels_and_query_tool_ test.CheckForXssFeatureTest) Test XSS check for panels and query tool ... ERROR ERROR runTest (pgadmin.feature_tests.xss_ checks_pgadmin_debugger_test. CheckDebuggerForXssFeatureTest ) Tests to check if Debugger is vulnerable to XSS ... ERROR ERROR runTest (pgadmin.feature_tests.xss_ checks_roles_control_test. CheckRoleMembershipControlFeat ureTest) Tests to check if Role membership control is vulnerable to XSS^[ ... Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ python_test_utils/test_utils. py", line 338, in create_role sql_query psycopg2.ProgrammingError: role "test_role" already exists Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ python_test_utils/test_utils. py", line 338, in create_role sql_query psycopg2.ProgrammingError: role "<h1>test</h1>" already exists ERROR ERROR ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.copy_ selected_query_results_ feature_test. CopySelectedQueryResultsFeatur eTest) Copy rows, column using button and keyboard shortcut ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/copy_selected_query_ results_feature_test.py", line 57, in runTest self._copies_rows() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/copy_selected_query_ results_feature_test.py", line 67, in _copies_rows pyperclip.copy("old clipboard contents") File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/pyperclip/__ init__.py", line 574, in lazy_load_stub_copy return copy(text) File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/pyperclip/__ init__.py", line 284, in __call__ raise PyperclipException(EXCEPT_MSG) pyperclip.PyperclipException: Pyperclip could not find a copy/paste mechanism for your system. For more information, please visit https://pyperclip.readthedocs. io/en/latest/introduction. html#not-implemented-error ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.pg_ datatype_validation_test. PGDataypeFeatureTest) Test checks for PG data-types output ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/pg_datatype_validation_ test.py", line 135, in runTest self._check_datatype() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/pg_datatype_validation_ test.py", line 169, in _check_datatype "contains(.,'{}')]".format( batch['datatype'][0]) File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.query_ tool_journey_test. QueryToolJourneyTest) Tests the path through the query tool ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/query_tool_journey_test. py", line 49, in runTest self._test_copies_rows() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/query_tool_journey_test. py", line 54, in _test_copies_rows pyperclip.copy("old clipboard contents") File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/pyperclip/__ init__.py", line 284, in __call__ raise PyperclipException(EXCEPT_MSG) pyperclip.PyperclipException: Pyperclip could not find a copy/paste mechanism for your system. For more information, please visit https://pyperclip.readthedocs. io/en/latest/introduction. html#not-implemented-error ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.query_ tool_tests. QueryToolFeatureTest) Query tool feature test ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/query_tool_tests.py", line 59, in runTest self._query_tool_explain_with_ verbose_and_cost() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/query_tool_tests.py", line 262, in _query_tool_explain_with_ verbose_and_cost self.page.find_by_id("btn- explain-costs").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 174, in find_by_id lambda driver: driver.find_element_by_id( element_id) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for element to exist ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.view_ data_dml_queries. CheckForViewDataTest) Validate Insert, Update operations in View/Edit data with given test data ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/view_data_dml_queries. py", line 114, in runTest self.page.add_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 55, in add_server self.find_by_partial_link_ text("Server...").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 185, in find_by_partial_link_text EC.element_to_be_clickable(( By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for link with text "Server..." ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.view_ data_dml_queries. CheckForViewDataTest) Validate Insert, Update operations in View/Edit data with given test data ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/base_feature_ test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/view_data_dml_queries. py", line 132, in after self.page.remove_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath( xpath) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for element to exist ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_file_manager_test. CheckFileManagerFeatureTest) Tests to check if File manager is vulnerable to XSS ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/base_feature_ test.py", line 43, in setUp self.before() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_file_manager_ test.py", line 37, in before self.page.add_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 55, in add_server self.find_by_partial_link_ text("Server...").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 185, in find_by_partial_link_text EC.element_to_be_clickable(( By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for link with text "Server..." ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_panels_and_query_tool_ test.CheckForXssFeatureTest) Test XSS check for panels and query tool ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_panels_and_ query_tool_test.py", line 57, in runTest self.page.add_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 55, in add_server self.find_by_partial_link_ text("Server...").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 185, in find_by_partial_link_text EC.element_to_be_clickable(( By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for link with text "Server..." ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_panels_and_query_tool_ test.CheckForXssFeatureTest) Test XSS check for panels and query tool ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/base_feature_ test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_panels_and_ query_tool_test.py", line 69, in after self.page.remove_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath( xpath) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for element to exist ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_pgadmin_debugger_test. CheckDebuggerForXssFeatureTest ) Tests to check if Debugger is vulnerable to XSS ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_pgadmin_ debugger_test.py", line 41, in runTest self.page.add_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 55, in add_server self.find_by_partial_link_ text("Server...").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 185, in find_by_partial_link_text EC.element_to_be_clickable(( By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for link with text "Server..." ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_pgadmin_debugger_test. CheckDebuggerForXssFeatureTest ) Tests to check if Debugger is vulnerable to XSS ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/base_feature_ test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_pgadmin_ debugger_test.py", line 46, in after self.page.remove_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath( xpath) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for element to exist ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_roles_control_test. CheckRoleMembershipControlFeat ureTest) Tests to check if Role membership control is vulnerable to XSS ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_roles_ control_test.py", line 37, in runTest self.page.add_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 55, in add_server self.find_by_partial_link_ text("Server...").click() File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 185, in find_by_partial_link_text EC.element_to_be_clickable(( By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for link with text "Server..." ============================== ============================== ========== ERROR: runTest (pgadmin.feature_tests.xss_ checks_roles_control_test. CheckRoleMembershipControlFeat ureTest) Tests to check if Role membership control is vulnerable to XSS ------------------------------ ------------------------------ ---------- Traceback (most recent call last): File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/base_feature_ test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/ pgadmin4/web/pgadmin/feature_ tests/xss_checks_roles_ control_test.py", line 42, in after self.page.remove_server(self. server) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath( xpath) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/ pgadmin4/web/regression/ feature_utils/pgadmin_page.py" , line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/ versions/python36/lib/python3. 6/site-packages/selenium/ webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions. TimeoutException: Message: Timed out waiting for element to exist ------------------------------ ------------------------------ ---------- Ran 11 tests in 441.881s FAILED (errors=13) ============================== ============================== ========== Test Result Summary ============================== ============================== ========== PostgreSQL 10: 2 tests passed 9 tests failed: CopySelectedQueryResultsFeatur eTest (Copy rows, column using button and keyboard shortcut) PGDataypeFeatureTest (Test checks for PG data-types output) QueryToolJourneyTest (Tests the path through the query tool) QueryToolFeatureTest (Query tool feature test) CheckForViewDataTest (Validate Insert, Update operations in View/Edit data with given test data) CheckFileManagerFeatureTest (Tests to check if File manager is vulnerable to XSS) CheckForXssFeatureTest (Test XSS check for panels and query tool) CheckDebuggerForXssFeatureTest (Tests to check if Debugger is vulnerable to XSS) CheckRoleMembershipControlFeat ureTest (Tests to check if Role membership control is vulnerable to XSS) 0 tests skipped ============================== ============================== ========== All the above tests fail in the following place:Also the tests will never end because firefox is waiting for the user to click "Leave the page" button.ThanksVictoria & JoaoHi Hackers,On Mon, Apr 23, 2018 at 9:07 PM, Anthony Emengo <aemengo@pivotal.io> wrote:We also tried running the tests with this patch. It didn't launch without some code changes and several tests were failing. We should really defer pulling this in until we have more robust results on FirefoxIn order to have the tests running we had to do the following change:diff --git a/web/regression/feature_utils/app_starter.py b/web/regression/feature_ utils/app_starter.py index 77b0400c..50d3e307 100644--- a/web/regression/feature_utils/app_starter.py +++ b/web/regression/feature_utils/app_starter.py @@ -42,10 +44,18 @@ class AppStarter:)self.driver.set_window_size(1280, 1024) - self.driver.get(- "http://" + self.app_config.DEFAULT_SERVER + ":" +- random_server_port- )+ # self.driver.implicitly_wait(60) ++ def launch_browser():+ try:+ self.driver.get(+ "http://" + self.app_config.DEFAULT_SERVER + ":" ++ random_server_port+ )+ except WebDriverException as e:+ time.sleep(5)+ launch_browser()+ launch_browser()This change was required because firefox was throwing an exception when we tried to get the address and the server was not running. We saw this behavior in Ubuntu.Yes I have faced the same problem in Ubuntu. As per suggestion by Dave I have added parameter in test_config.json.in and also include the above code given by Anthony.Attached is the modified patch.- Anthony and JoaoOn Mon, Apr 23, 2018 at 9:12 AM, Dave Page <dpage@pgadmin.org> wrote:HiOn Mon, Apr 23, 2018 at 2:05 PM, Akshay Joshi <akshay.joshi@enterprisedb.com> wrote: Hi Hackers,I have added support for running feature tests against FireFox. For that user will have to download gecko driver from https://github.com/mozilla/geckodriver/releases and follow the below steps:
- Extract the gecko driver.
- Run chmod +x geckodriver.
- Either copy the geckodriver to /usr/local/bin or the path of the geckodriver must be specified in PATH.
- Apply the attached patch.
- Change the parameter "DEFAULT_TEST_BROWSER = 'Firefox' "
- Start the feature test.
The config option needs to be in test_config.json. We don't really want test-specific config options in config.py (arguably, TEST_SQLITE_PATH shoudn't be there either, as it's useless for end users).I'm surprised to not see any updates to tests. When I tried firefox manually a few weeks back, it was failing at lot.--Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachment
pgadmin-hackers by date: