Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4] - Mailing list pgadmin-hackers
From | Harshal Dhumal |
---|---|
Subject | Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4] |
Date | |
Msg-id | CAFiP3vxfk3fwuYRn51EsHBaH67WC55aNx+yS6e__5gy5Vf9Kow@mail.gmail.com Whole thread Raw |
In response to | Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4] (Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com>) |
Responses |
Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]
|
List | pgadmin-hackers |
Hi,
Please find updated patch for unicode issue on python 2.7I have tested with below scenarios for all nodes (except database and Login/Group Role as these are stored in shared catalogs as for now we are not considering encoding issues for these two nodes)
scenarios: Create and Update nodes
Note: Apply patches which I have just sent (with subject pgAdmin4 minor issues [pgAdmin4]) before applying this patch.
--
Harshal Dhumal
Software Engineer
On Fri, Feb 17, 2017 at 4:22 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi Harshal,Python3.51) I installed PG_9.4 with locale: Arabic, Saudi Arabia,- When I try to create new database with name "السلام" -> with 'template 0' as template , it fails to open, see screenshot- Backup/Restore fails, refer screenshotI tested INSET/UPDATE/DELETE and other common operations and seems to be working.Attaching screenshots.2) I installed PG_9.5 with locale: Chinese, China- I am not able to create new DB with Chinese name same as above mentionedSame as (1), most parts are working with patchAttaching screenshots.3) I installed PG_9.5 with locale: Japanese, Japan- I am not able to create new DB with Japanese name same as above mentionedSame as (1), most parts are working with patchAttaching screenshots.Python2.7 with 3 server mentioned aboveWorks fine most the places but also fails at some places with Unicode errors, As I observed it fails when we receive the data from DB server (by running properties.sql) and we try to render the data in template as we received from DB server.Screenshots attached.Common Exceptions:{{conn|qtIdent(c.name)}} {{ GET_TYPE.CREATE_TYPE_SQL(conn, c.cltype, c.attlen, c.attprecision, c.hasSqrBracket) }}{% if c.collspcname %} COLLATE {{c.collspcname}}{% endif %}{% if c.attnotnull %} NOT NULL{% endif %}{% if c.defval %} DEFAULT {{c.defval}}{% endif %}UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 10: ordinal not in range(128)Traceback (most recent call last):File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 2000, in __call__ return self.wsgi_app(environ, start_response)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1991, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1567, in handle_exception reraise(exc_type, exc_value, tb)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request()File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1641, in full_dispatch_request rv = self.handle_user_exception(e)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1544, in handle_user_exception reraise(exc_type, exc_value, tb)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1639, in full_dispatch_request rv = self.dispatch_request()File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/app.py", line 1625, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/views.py", line 84, in view return self.dispatch_request(*args, **kwargs)File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/ utils.py", line 235, in dispatch_request return method(*args, **kwargs)File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/ server_groups/servers/ databases/schemas/functions/__ init__.py", line 341, in wrap return f(*args, **kwargs)File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/ server_groups/servers/ databases/schemas/functions/__ init__.py", line 1021, in sql query_for="sql_panel")File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/templating.py", line 134, in render_template context, ctx.app)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ flask/templating.py", line 116, in _render rv = template.render(context)File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ jinja2/environment.py", line 969, in render return self.environment.handle_exception(exc_info, True) File "/home/murtuza/projects/venv2/lib/python2.7/site-packages/ jinja2/environment.py", line 742, in handle_exception reraise(exc_type, exc_value, tb)File "/home/murtuza/projects/pgadmin4/web/pgadmin/browser/ server_groups/servers/ databases/schemas/functions/ templates/function/pg/sql/9.5_ plus/create.sql", line 7, in top-level template code CREATE{% if query_type is defined %}{{' OR REPLACE'}}{% endif %} FUNCTION {{func_def}}UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)--Regards,On Thu, Feb 9, 2017 at 4:34 PM, Dave Page <dpage@pgadmin.org> wrote:Harshal,
I had a brief discussion with Ashesh about this. He's going to setup a
call next week for us to discuss a plan of action for eliminating any
remaining encoding issues including this one. I'm putting this on hold
until then.
Thanks.
On Mon, Feb 6, 2017 at 2:54 PM, Harshal Dhumal
<harshal.dhumal@enterprisedb.com> wrote:
> Hi Dave,
>
> --
> Harshal Dhumal
> Software Engineer
>
> EnterpriseDB India: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Feb 6, 2017 at 6:48 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Hi
>>
>> On Mon, Feb 6, 2017 at 12:57 PM, Harshal Dhumal
>> <harshal.dhumal@enterprisedb.com> wrote:
>> > Hi,
>> >
>> > Please find attached patch for RM 1983.
>> >
>> > This issue only occurs when database encoding is other than utf-8
>> >
>> > Also other issue was when we use connection of database with encoding
>> > other
>> > than utf-8 to retrieve data from cluster table/s which has encoding
>> > utf-8
>> > (e.g. pg_database) then data was not decoded properly.
>>
>> The code makes an assumption that pg_database is always utf-8 encoded.
>> I don't believe that is correct - I believe it's the encoding used in
>> the database from which the new database was created. The general
>> advice is that users should avoid using non-7bit ASCII characters in
>> shared catalogs, e.g. databases and comments etc.
>
>
> Ok.
>
> Let me split this into two issues:
> i) RM1983 for which I have attached updated patch. (basically I removed
> changes related to decode data retried from pg_database when connection
> encoding is other than utf-8)
> ii) Support to allow user to use non-&bit ASCII characters in shared
> catalogs with the help of pgAdmin4.
>
> Regarding your statement about pg_database "I believe it's the encoding used
> in the database from which the new database was created.". I found it
> little-bit confusing for me (correct me if i'm wrong); As mentioned here
> there is only one copy of pg_database per cluster. So I assume pg_database
> is created when we initialize database cluster and not when we create new
> database.
>
>
>>
>>
>> See
>> https://www.postgresql.org/message-id/flat/20160216163833. GF31273%40momjian.us# 20160216163833.GF31273@ momjian.us
>> for more info for example.
>>
>> Did pgAdmin 3 just assume it was UTF-8? I suspect it did - and that
>> just happened to work in most cases.
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>
>--
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers
Attachment
pgadmin-hackers by date: