Re: [pgadmin4][Patch]: Test cases for the backup module - Mailing list pgadmin-hackers

Hi

On Tue, Jun 5, 2018 at 4:39 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

On Mon, Jun 4, 2018 at 8:41 PM, Joao De Almeida Pereira <jdealmeidapereira@pivotal.io> wrote:

Hi Khushboo,

Some tests are failing in greenplum:

https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/108

The piece of code responsible for the error is:

if server['default_binary_paths'] is not None:
    test_utils.set_preference(server['default_binary_paths'])

    config.DEFAULT_BINARY_PATHS = {
        "pg": str(server['default_binary_paths']['pg']),
        "ppas": str(server['default_binary_paths']['ppas']),
        "gpdb": ""
    }

Can you send me the test_config.json file?  The above code sets the paths to the SQLite database and through the logs couldn't figure out the exact failure. 

It seems clear from the code shown that it's not setting the binary paths for gpdb database servers. Shouldn't it be something like:

    config.DEFAULT_BINARY_PATHS = {
        "pg": str(server['default_binary_paths']['pg']),
        "ppas": str(server['default_binary_paths']['ppas']),
        "gpdb": str(server['default_binary_paths']['gpdb'])
    }

    test_utils.set_preference(server['default_binary_paths'])

 

test_backup_utils.py file name is misleading, these are not tests, are helpers.

​

Thanks

Victoria & Joao

On Mon, Jun 4, 2018 at 1:36 AM Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

On Sat, Jun 2, 2018 at 3:01 AM, Dave Page <dpage@pgadmin.org> wrote:

Hi

This looks good, except that it's leaving the test_restore_database behind. Can we clean that up please?

PFA updated patch. 

Thanks.

On Fri, Jun 1, 2018 at 7:06 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi Victoria,

Thanks for reviewing the patch.

The tests were failing due to the latest commit #2b4605a9d390cb44e5dfe9967c3adf2b28d04f1f  - Ensure backup/restore/maintenance work via SSH tunnels. Fixes #3355

I have fixed the issues and attached the updated patch.

Thanks,

Khushboo 

On Thu, May 31, 2018 at 10:00 PM, Victoria Henry <vhenry@pivotal.io> wrote:

Hi there,

We've been noticing some issues with the tests on both our CI and local Mac workstations.

1. When the following code blocks are invoked - we get plenty of app.context() issues. It must not be valid when running tests.

​

from pgadmin.utils.driver import get_driver
driver = get_driver(PG_DEFAULT_DRIVER)
manager = driver.connection_manager(self.sid)

host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
port = manager.local_bind_port if manager.use_ssh_tunnel else s.port 

2. When we finally enable 

"default_binary_paths": {

in our test_config, we get more failing tests that look like:

======================================================================
FAIL: runTest (pgadmin.tools.restore.tests.test_restore_create_job_unit_test.RestoreCreateJobTest)
When restore object with option - Miscellaneous
----------------------------------------------------------------------
Traceback (most recent call last): File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/unittest/mock.py", line 1179, in patched   return func(*args, **keywargs) File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py", line 295, in runTest   self.assertEquals(response.status_code, 200)
AssertionError: 410 != 200

And

When restore object with the sections options ... 2018-05-31 12:24:42,988: ERROR    pgadmin:    illegal environment variable name
Traceback (most recent call last): File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/__init__.py", line 352, in create_restore_job   manager.export_password_env(p.id) File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py", line 365, in export_password_env   os.environ[str(env)] = password File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/os.py", line 675, in __setitem__   self.putenv(key, value)
ValueError: illegal environment variable name
FAIL

​

Sincerely,

Victoria && Anthony

On Thu, May 31, 2018 at 1:16 AM Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi,

Please find the attached updated patch with the fixes.

The test cases were only failing on MAC not on Linux.

Thanks,

Khushboo

On Wed, May 30, 2018 at 10:13 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

On Wed, May 30, 2018 at 1:05 AM, Dave Page <dpage@pgadmin.org> wrote:

Hi

On Mon, May 28, 2018 at 8:09 AM, Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi,

please find the attached updated patch for the test cases of Backup, Restore and Maintenance modules which includes:

1. Unit test cases

2. End to end regression test cases 

3. Feature test cases

Thanks. I've yet to be able to run the feature tests successfully. Here's what I've found so far:

1) DEFAULT_BINARY_PATHS should be default_binary_paths in the JSON config file.

Will do. 

2) I've hit screensize related issues:

======================================================================

ERROR: runTest (pgadmin.feature_tests.pg_utilities_maintenance_test.PGUtilitiesMaintenanceFeatureTest)

Test for PG maintenance: database

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py", line 56, in runTest

    self._open_maintenance_dialogue()

  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py", line 75, in _open_maintenance_dialogue

    "*[.='" + self.table_name + "']/../*[@class='aciTreeItem'"

  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click

    self._execute(Command.CLICK_ELEMENT)

  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute

    return self._parent.execute(command, params)

  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute

    self.error_handler.check_response(response)

  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response

    raise exception_class(message, screen, stacktrace)

WebDriverException: Message: unknown error: Element <span class="aciTreeItem">...</span> is not clickable at point (223, 604). Other element would receive the click: <div class="wcFrameCenter wcPanelBackground wcScrollableX wcScrollableY" style="left: 0px; right: 0px; bottom: 0px;">...</div>

  (Session info: chrome=66.0.3359.181)

  (Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.12.6 x86_64)

3) One time the test did start, but then I saw this failure:

======================================================================

ERROR: runTest (pgadmin.feature_tests.pg_utilities_backup_restore_test.PGUtilitiesBackupFeatureTest)

Test for PG utilities - Backup and Restore

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py", line 93, in runTest

    self.page.fill_input_by_field_name("file", "test_backup_file")

  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 211, in fill_input_by_field_name

    self.wait_for_input_field_content(field_name, field_content)

  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 251, in wait_for_input_field_content

    "field to contain '" + str(content) + "'", input_field_has_content

  File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 337, in _wait_for

    "Timed out waiting for " + waiting_for_message

  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until

    raise TimeoutException(message, screen, stacktrace)

TimeoutException: Message: Timed out waiting for field to contain 'test_backup_file'

(with screenshot attached)

Thanks.

I have ran the feature tests with multiple servers many times but didn't get a single failure.

I have asked Akshay to run on his machine, let see what he gets.

 

Thanks,

Khushboo

On Wed, Apr 25, 2018 at 9:40 PM, Joao De Almeida Pereira <jdealmeidapereira@pivotal.io> wrote:

Hi Khushboo,

We reviewed the patch and it is very nice to see some more coverage in this area. Good job :D

We passed the tests through our CI the feature tests are not passing, but the linter fails:

./pgadmin/feature_tests/pg_utilities_backup_test.py:37: [E501] line too long (91 > 79 characters)
./pgadmin/feature_tests/pg_utilities_backup_test.py:53: [E501] line too long (104 > 79 characters)
./pgadmin/feature_tests/pg_utilities_backup_test.py:59: [E501] line too long (85 > 79 characters)
./pgadmin/feature_tests/pg_utilities_backup_test.py:62: [E501] line too long (96 > 79 characters)
./pgadmin/feature_tests/pg_utilities_backup_test.py:63: [E501] line too long (91 > 79 characters)
./pgadmin/feature_tests/pg_utilities_backup_test.py:70: [E501] line too long (118 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:37: [E121] continuation line under-indented for hanging indent
./pgadmin/tools/backup/tests/test_backup_message.py:48: [E122] continuation line missing indentation or outdented
./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
./pgadmin/tools/backup/tests/test_backup_message.py:51: [E501] line too long (91 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:52: [E501] line too long (94 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:53: [E501] line too long (108 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:81: [E501] line too long (113 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:82: [E501] line too long (94 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:83: [E501] line too long (108 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:111: [E501] line too long (100 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:113: [E501] line too long (94 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:114: [E501] line too long (108 > 79 characters)
./pgadmin/tools/backup/tests/test_backup_message.py:147: [E501] line too long (93 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:40: [E121] continuation line under-indented for hanging indent
./pgadmin/tools/backup/tests/test_batch_process.py:51: [E122] continuation line missing indentation or outdented
./pgadmin/tools/backup/tests/test_batch_process.py:135: [E501] line too long (80 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:137: [E501] line too long (83 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:138: [E122] continuation line missing indentation or outdented
./pgadmin/tools/backup/tests/test_batch_process.py:139: [E122] continuation line missing indentation or outdented
./pgadmin/tools/backup/tests/test_batch_process.py:140: [E122] continuation line missing indentation or outdented
./pgadmin/tools/backup/tests/test_batch_process.py:191: [E501] line too long (81 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:203: [E501] line too long (80 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:204: [E128] continuation line under-indented for visual indent
./pgadmin/tools/backup/tests/test_batch_process.py:204: [E501] line too long (94 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:205: [E128] continuation line under-indented for visual indent
./pgadmin/tools/backup/tests/test_batch_process.py:205: [E501] line too long (94 > 79 characters)
./pgadmin/tools/backup/tests/test_batch_process.py:216: [W391] blank line at end of file
./pgadmin/tools/backup/tests/test_create_backup_job.py:296: [E501] line too long (97 > 79 characters)
./pgadmin/tools/backup/tests/test_create_backup_job.py:317: [E303] too many blank lines (2)
./pgadmin/tools/backup/tests/test_create_backup_job.py:336: [E501] line too long (84 > 79 characters)
./pgadmin/tools/backup/tests/test_create_backup_job.py:371: [W391] blank line at end of file
2       E121 continuation line under-indented for hanging indent
5       E122 continuation line missing indentation or outdented
2       E128 continuation line under-indented for visual indent
2       E251 unexpected spaces around keyword / parameter equals
1       E303 too many blank lines (2)
24      E501 line too long (91 > 79 characters)
2       W391 blank line at end of file
38

For the feature tests, we realized we had to update the configuration, and we did that, but we get the following error attached. We spent some time trying to understand the problem but we were not successful.

Codewise:

- We just found some One Letter Variables in the code...

- Looks like there is a bug report in this area of the code and we do not have coverage for it: https://redmine.postgresql.org/issues/3232

  Looks like in some of the unit tests we only have happy path tests, maybe we should see if there are any sad paths that also need coverage.

The configuration change, maybe need to be updated. When we install multiple versions of postgres the binaries live in `/usr/lib/postgresql/{{db_version}}/bin`, which makes us think that this configuration should live near the server configuration, maybe? Also to maintain coherency on the naming maybe we should make it all lower case.

Just as an aside, you can add the gpdb configuration as well in you patch.

Thanks

Victoria & Joao

On Wed, Apr 25, 2018 at 5:20 AM Khushboo Vashi <khushboo.vashi@enterprisedb.com> wrote:

Hi,

Please find the attached patch which covers test cases for the backup module (RM #3206).

1. Unit test cases

2. End to end regression test cases 

3. Feature test cases

Thanks,

Khushboo

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company