Postgres Pro
Downloads
Home   >   mailing lists

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database. - Mailing list pgadmin-hackers

From Dave Page
Subject Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.
Date
Msg-id CA+OCxowxJghshFjXCtOUiRa0qqS93pC4m-bPRhdb7Q3AWe+HnQ@mail.gmail.com
Whole thread Raw
In response to Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.  (Aditya Toshniwal <aditya.toshniwal@enterprisedb.com>)
Responses Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.
List pgadmin-hackers
Tree view
Hi

On Mon, Jun 18, 2018 at 2:14 PM, Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:
Hi Hackers,

Attached is the updated patch which includes the fix for Download CSV fail in SQL_ASCII database, which is RM3250 
This should fix RM3289 and RM3250. As they interrelated, sending the combined patch.
Kindly review.

This is definitely looking better - both view and save now work as expected. However, using the test data the I posted upthread, if I try to edit a value (in this case by adding a couple of chars to the end of the data in row 2) I get:

2018-06-18 16:41:40,895: SQL pgadmin: Execute (void) for server #1 - DB:ascii (Query-id: 3093186):
UPDATE public.ascii SET
data = %(data)s::text WHERE
id = '2';
2018-06-18 16:41:41,027: INFO werkzeug: 127.0.0.1 - - [18/Jun/2018 16:41:41] "POST /sqleditor/save/2805058 HTTP/1.1" 500 -
2018-06-18 16:41:41,042: ERROR werkzeug: Error on request:
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", line 270, in run_wsgi
    execute(self.server.app)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", line 258, in execute
    application_iter = app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py", line 792, in decorated_view
    return func(*args, **kwargs)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py", line 776, in save
    default_conn)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqleditor/command.py", line 829, in save
    item['sql'], item['data'])
  File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", line 975, in execute_void
    self.__internal_blocking_execute(cur, query, params)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", line 629, in __internal_blocking_execute
    cur.execute(query, params)
  File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/cursor.py", line 176, in execute
    return _cursor.execute(self, query, params)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 19-21: ordinal not in range(128)
 

On Fri, Jun 15, 2018 at 2:33 PM, Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:
Hi Dave,

On Fri, Jun 15, 2018 at 2:08 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi

On Thu, Jun 14, 2018 at 7:05 AM, Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:
I am sorry I missed the attachment. :(
PFA.

It looks like the encoding names are getting munged somewhere. I see you've accounted for that to some degree in connection.py (you have both SQL_ASCII/MULE_INTERNAL and SQLASCII/MULEINTERNAL), however it doesn't seem to be enough as I'm getting the following error when trying to download CSV from the query tool. Can we ensure that conn.encoding contains an un-munged value at all times, or is that coming from psycopg2?
​That is done by pyscopg2 and conn.encoding is a psycopg2 connection property.​
 

2018-06-15 09:32:28,799: INFO werkzeug: 127.0.0.1 - - [15/Jun/2018 09:32:28] "GET /sqleditor/query_tool/download/2732923?query=SELECT%20*%20FROM%20public.sql_ascii%0AORDER%20BY%20id%20ASC%20&filename=sql_ascii.csv HTTP/1.1" 500 -
2018-06-15 09:32:28,801: ERROR werkzeug: Error on request:
Traceback (most recent call last):
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", line 270, in run_wsgi
    execute(self.server.app)
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py", line 260, in execute
    for data in application_iter:
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wsgi.py", line 870, in __next__
    return self._next()
  File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/wrappers.py", line 82, in _iter_encoded
    for item in iterable:
  File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", line 752, in gen
    column_name = column_name.decode(conn_encoding)
LookupError: unknown encoding: SQLASCII
 
​This is because there is code bug here. Below is code used to decode a column name. Connection encoding and python encoding are two different things. Python does not know what SQLASCII is. This will work with UTF-8 because python has decoder with same name. I tried to download CSV with the original code without changes and it fails there too. I will fix this and will send the updated patch. I should have checked this.
conn_encoding = cur.connection.encoding
column_name = column_name.decode(conn_encoding)​
 

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



--
Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"



--
Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

pgadmin-hackers by date:

Previous
From: Dave Page
Date:
Subject: Re: [pgadmin4][Patch]: RM #3361 Create database dialog does not showtemplate0 and template1 database in Template dropdown
Next
From: Akshay Joshi
Date:
Subject: Re: [pgadmin4][Patch]: Feature #3388 Allow to set the connectiontimeout on the server properties dialog