Re: [pgAdmin4][Patch] Feature #3270 Add support for running regression tests against Firefox - Mailing list pgadmin-hackers
Hi Akshay,
Linter fails and tests are failing when we run the patch on our machine using Firefox. The linter output is:
./regression/runtests.py:184: [E125] continuation line with same indent as next logical line./regression/runtests.py:184: [E501] line too long (80 > 79 characters)1 E125 continuation line with same indent as next logical line1 E501 line too long (80 > 79 characters)2
The tests output is:
=============Running the test cases for 'PostgreSQL 10'=============
runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
Copy rows, column using button and keyboard shortcut ... ERROR
runTest (pgadmin.feature_tests.keyboard_shortcut_test.KeyboardShortcutFeatureTest)
Test for keyboard shortcut ... Executing shortcut: File main menu...OK
Executing shortcut: Object main menu...OK
ok
runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
Test checks for PG data-types output ... ERROR
runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
Tests the path through the query tool ... ERROR
runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
Query tool feature test ...
On demand query result...
On demand result set on scrolling... OK.
On demand result set on grid select all... OK.
On demand result set on column select all... OK.
Explain query with verbose and cost... ERROR
runTest (pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)
Test table DDL generation ... ok
runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
Validate Insert, Update operations in View/Edit data with given test data ... ERROR
ERROR
runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
Tests to check if File manager is vulnerable to XSS ... ERROR
runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
Test XSS check for panels and query tool ... ERROR
ERROR
runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
Tests to check if Debugger is vulnerable to XSS ... ERROR
ERROR
runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
Tests to check if Role membership control is vulnerable to XSS^[ ... Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role sql_query
psycopg2.ProgrammingError: role "test_role" already exists
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role sql_query
psycopg2.ProgrammingError: role "<h1>test</h1>" already exists
ERROR
ERROR
======================================================================
ERROR: runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
Copy rows, column using button and keyboard shortcut
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 57, in runTest self._copies_rows() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 67, in _copies_rows pyperclip.copy("old clipboard contents") File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 574, in lazy_load_stub_copy return copy(text) File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__ raise PyperclipException(EXCEPT_MSG)
pyperclip.PyperclipException: Pyperclip could not find a copy/paste mechanism for your system. For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
======================================================================
ERROR: runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
Test checks for PG data-types output
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 135, in runTest self._check_datatype() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 169, in _check_datatype "contains(.,'{}')]".format(batch['datatype'][0]) File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
======================================================================
ERROR: runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
Tests the path through the query tool
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 49, in runTest self._test_copies_rows() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 54, in _test_copies_rows pyperclip.copy("old clipboard contents") File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__ raise PyperclipException(EXCEPT_MSG)
pyperclip.PyperclipException: Pyperclip could not find a copy/paste mechanism for your system. For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
======================================================================
ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
Query tool feature test
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 59, in runTest self._query_tool_explain_with_verbose_and_cost() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 262, in _query_tool_explain_with_verbose_and_cost self.page.find_by_id("btn-explain-costs").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 174, in find_by_id lambda driver: driver.find_element_by_id(element_id) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
======================================================================
ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
Validate Insert, Update operations in View/Edit data with given test data
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 114, in runTest self.page.add_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server self.find_by_partial_link_text("Server...").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
======================================================================
ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
Validate Insert, Update operations in View/Edit data with given test data
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 132, in after self.page.remove_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath(xpath) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
Tests to check if File manager is vulnerable to XSS
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 43, in setUp self.before() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_file_manager_test.py", line 37, in before self.page.add_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server self.find_by_partial_link_text("Server...").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
Test XSS check for panels and query tool
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 57, in runTest self.page.add_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server self.find_by_partial_link_text("Server...").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
Test XSS check for panels and query tool
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 69, in after self.page.remove_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath(xpath) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
Tests to check if Debugger is vulnerable to XSS
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 41, in runTest self.page.add_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server self.find_by_partial_link_text("Server...").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
Tests to check if Debugger is vulnerable to XSS
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 46, in after self.page.remove_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath(xpath) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
Tests to check if Role membership control is vulnerable to XSS
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 37, in runTest self.page.add_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server self.find_by_partial_link_text("Server...").click() File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text)) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
======================================================================
ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
Tests to check if Role membership control is vulnerable to XSS
----------------------------------------------------------------------
Traceback (most recent call last): File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown self.after() File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 42, in after self.page.remove_server(self.server) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server "' and @class='aciTreeItem']") File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath lambda driver: driver.find_element_by_xpath(xpath) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for "Timed out waiting for " + waiting_for_message File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
----------------------------------------------------------------------
Ran 11 tests in 441.881s
FAILED (errors=13)
======================================================================
Test Result Summary
======================================================================
PostgreSQL 10:
2 tests passed 9 tests failed: CopySelectedQueryResultsFeatureTest (Copy rows, column using button and keyboard shortcut) PGDataypeFeatureTest (Test checks for PG data-types output) QueryToolJourneyTest (Tests the path through the query tool) QueryToolFeatureTest (Query tool feature test) CheckForViewDataTest (Validate Insert, Update operations in View/Edit data with given test data) CheckFileManagerFeatureTest (Tests to check if File manager is vulnerable to XSS) CheckForXssFeatureTest (Test XSS check for panels and query tool) CheckDebuggerForXssFeatureTest (Tests to check if Debugger is vulnerable to XSS) CheckRoleMembershipControlFeatureTest (Tests to check if Role membership control is vulnerable to XSS) 0 tests skipped
======================================================================
All the above tests fail in the following place:
Also the tests will never end because firefox is waiting for the user to click "Leave the page" button.
Thanks
Victoria & Joao
On Tue, Apr 24, 2018 at 2:26 AM Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:
Hi Hackers,On Mon, Apr 23, 2018 at 9:07 PM, Anthony Emengo <aemengo@pivotal.io> wrote:We also tried running the tests with this patch. It didn't launch without some code changes and several tests were failing. We should really defer pulling this in until we have more robust results on FirefoxIn order to have the tests running we had to do the following change:diff --git a/web/regression/feature_utils/app_starter.py b/web/regression/feature_utils/app_starter.pyindex 77b0400c..50d3e307 100644--- a/web/regression/feature_utils/app_starter.py+++ b/web/regression/feature_utils/app_starter.py@@ -42,10 +44,18 @@ class AppStarter:)self.driver.set_window_size(1280, 1024)- self.driver.get(- "http://" + self.app_config.DEFAULT_SERVER + ":" +- random_server_port- )+ # self.driver.implicitly_wait(60)++ def launch_browser():+ try:+ self.driver.get(+ "http://" + self.app_config.DEFAULT_SERVER + ":" ++ random_server_port+ )+ except WebDriverException as e:+ time.sleep(5)+ launch_browser()+ launch_browser()This change was required because firefox was throwing an exception when we tried to get the address and the server was not running. We saw this behavior in Ubuntu.Yes I have faced the same problem in Ubuntu. As per suggestion by Dave I have added parameter in test_config.json.in and also include the above code given by Anthony.Attached is the modified patch.- Anthony and JoaoOn Mon, Apr 23, 2018 at 9:12 AM, Dave Page <dpage@pgadmin.org> wrote:HiOn Mon, Apr 23, 2018 at 2:05 PM, Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:Hi Hackers,I have added support for running feature tests against FireFox. For that user will have to download gecko driver from https://github.com/mozilla/geckodriver/releases and follow the below steps:
- Extract the gecko driver.
- Run chmod +x geckodriver.
- Either copy the geckodriver to /usr/local/bin or the path of the geckodriver must be specified in PATH.
- Apply the attached patch.
- Change the parameter "DEFAULT_TEST_BROWSER = 'Firefox' "
- Start the feature test.
The config option needs to be in test_config.json. We don't really want test-specific config options in config.py (arguably, TEST_SQLITE_PATH shoudn't be there either, as it's useless for end users).I'm surprised to not see any updates to tests. When I tried firefox manually a few weeks back, it was failing at lot.--Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachment
pgadmin-hackers by date: