Re: [GSoC][New Feature] Editable and Read-only Columns - Mailing list pgadmin-hackers

Hi Yosry,

It's failing for me as well. I debugged and found that, sometimes the grid render is delayed (in microseconds) after query execution. Because of which, the cell you had selected using xpath is actually the older one. And when you're trying to set the value the cell no longer exists as the new cell is rendered now. That's why stale element exception. Adding below code solved the problem for me. Attached is the patch.

import time
time.sleep(0.5)
for column_index, should_be_editable in enumerated_should_be_editable:

On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93@gmail.com> wrote:

Hi Dave,

I am now using the same versions of Python, Chrome and chromedriver. The tests still pass every time. Are you sure you applied the latest patch? I re-created the patch as a sanity check, please find it attached.

I tried running the specific test a lot of times, passes every time. The only difference is that I am running Ubuntu, otherwise, everything is the same.

On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage@pgadmin.org> wrote:

Hi

I've run it again 4 more times and it fails consistently for me. I'm running on macOS 10.14.5 with Chrome 76.0.3809.100.

On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93@gmail.com> wrote:

Hi Dave,

The tests pass on my device normally. I am using Python 3.6 on Ubuntu. I have ran all tests using "make check" and they all succeeded. I also ran feature_tests/query_tool_journey_test.py multiple times on its own using "cd web && python regression/runtests.py --pkg feature_tests --modules query_tool_journey_test" and it passed successfully. Could you try again?

Generally, I have found that StaleElementReferenceException occurs intermittently on various feature tests.

On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage@pgadmin.org> wrote:

Thanks. Unfortunately this is failing the tests:

======================================================================
ERROR: runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
Tests the path through the query tool
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 85, in runTest
    self._test_updatable_resultset()
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 254, in _test_updatable_resultset
    discard_changes_modal=True)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 417, in _check_query_results_editable
    is_editable = self._check_cell_editable(column_index)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 430, in _check_cell_editable
    cell_value = int(cell_el.text)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 76, in text
    return self._execute(Command.GET_ELEMENT_TEXT)['value']
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute
    return self._parent.execute(command, params)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    self.error_handler.check_response(response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=76.0.3809.100)
  (Driver info: chromedriver=76.0.3809.126 (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809@{#1024}),platform=Mac OS X 10.14.5 x86_64)

Without the patch, the tests pass.

On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93@gmail.com> wrote:

Please find an updated patch attached.

On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage@pgadmin.org> wrote:

Except it now doesn't apply as the async connection/rendering patch was in the queue right ahead of it :-(.

Can I get a rebased patch please?

On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:

Excellent !! Looks good to me.

On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <yosrym93@gmail.com> wrote:

 I updated the patch for backwards compatibility anyway.

On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage@pgadmin.org> wrote:

On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <yosrym93@gmail.com> wrote:

Hi Aditya,

On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:

Hi Yosry,

The previously mentioned issues seems to be fixed. Below are few issues:

1) On Python 2.7, I get below error when opening query tool:

2019-08-23 10:49:09,329: ERROR flask.app: Object of type buffer is not JSON serializable
Traceback (most recent call last):
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py", line 1544, in get_query_history
    return QueryHistory.get(current_user.id, trans_obj.sid, conn.db)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py", line 21, in get
    'result': [rec.query_info for rec in result]
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line 75, in make_json_response
    separators=(',', ':'), encoding=encoding),
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py", line 399, in dumps
    **kw).encode(obj)
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py", line 296, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py", line 378, in iterencode
    return _iterencode(o, 0)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line 30, in default
    return json.JSONEncoder.default(self, obj)
  File "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py", line 273, in default
    o.__class__.__name__)
TypeError: Object of type buffer is not JSON serializable

This error exists on the master branch on Python 2.7, it is not caused by the patch. It is also not a regression of my previous Query History Enhancement patch, I checked.

 

2) The patch is not compatible with older pyscopg2 (2.7.7). I get below error in View/Edit data.

2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
Traceback (most recent call last):
  File "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py", line 435, in poll
    columns = trans_obj.get_columns_types(conn)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py", line 688, in get_columns_types
    table_oid=table_oid)
  File "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py", line 38, in get_columns_types
    if row['oid'] == col['table_column']:
KeyError: 'table_column'

I recall Dave mentioning that it is okay to use psycopg 2.8+ features and that it was a requirement of pgAdmin 4 now (clearly mentioned in requirements.txt too).

I did - then we had the whole table_oid issue on Debian/Ubuntu :-(

 

However, I updated the patch for backwards compatibility. Please find an updated patch attached.

--

Yosry Muhammad Yosry

Computer Engineering student,

The Faculty of Engineering,

Cairo University (2021).

Class representative of CMP 2021.

https://www.linkedin.com/in/yosrym93/

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Yosry Muhammad Yosry

Computer Engineering student,

The Faculty of Engineering,

Cairo University (2021).

Class representative of CMP 2021.

https://www.linkedin.com/in/yosrym93/

--

Thanks and Regards,

Aditya Toshniwal

Software Engineer | EnterpriseDB India | Pune

"Don't Complain about Heat, Plant a TREE"

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Yosry Muhammad Yosry

Computer Engineering student,

The Faculty of Engineering,

Cairo University (2021).

Class representative of CMP 2021.

https://www.linkedin.com/in/yosrym93/

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Yosry Muhammad Yosry

Computer Engineering student,

The Faculty of Engineering,

Cairo University (2021).

Class representative of CMP 2021.

https://www.linkedin.com/in/yosrym93/

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--

Yosry Muhammad Yosry

Computer Engineering student,

The Faculty of Engineering,

Cairo University (2021).

Class representative of CMP 2021.

https://www.linkedin.com/in/yosrym93/

--

Thanks and Regards,

Aditya Toshniwal

Software Engineer | EnterpriseDB India | Pune

"Don't Complain about Heat, Plant a TREE"