Thread: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

[pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Surinder Kumar

Date:

12 July 2016, 12:58:21

What is happening in code?

Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:

Database name should be encoded in 'utf-8' before storing to database.

This resolved the issue.

I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.

https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?

If it looks good, we can do the same changes in case of other nodes.

Thanks,

Surinder Kumar

Attachment

RM#1419.patch

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Murtuza Zabuawala

Date:

12 July 2016, 13:14:13

Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,

Murtuza

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Surinder Kumar

Date:

12 July 2016, 13:18:54

On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.

No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Surinder Kumar

Date:

12 July 2016, 13:27:55

On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

It is reproducible for me on

Mac OS X Yosemite version 10.10.2

Python version: 2.7

Browsers: Chrome & Firefox.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Murtuza Zabuawala

Date:

12 July 2016, 13:28:39

May be because of Python2.7 but if you try in Python3. It works.

we can use PEP 0263 to fix this issue. // # -*- coding: utf-8 -*-

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Ashesh Vashi

Date:

12 July 2016, 13:37:49

On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

May be because of Python2.7 but if you try in Python3. It works.

we can use PEP 0263 to fix this issue. // # -*- coding: utf-8 -*-

Isn't this about the source code encoding, and not about the input encoding?

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Surinder Kumar

Date:

12 July 2016, 13:49:10

On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
May be because of Python2.7 but if you try in Python3. It works.

we can use PEP 0263 to fix this issue. // # -*- coding: utf-8 -*-

I tried but it didn't worked.

It gives me error:

Traceback (most recent call last):

File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 639, in msql

status, res = self.get_sql(gid, sid, data, did)

File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 686, in get_sql

SQL += self.get_new_sql(gid, sid, data, did)

File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 721, in get_new_sql

data=data, conn=self.conn

File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template

context, ctx.app)

File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 110, in _render

rv = template.render(context)

File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render

return self.environment.handle_exception(exc_info, True)

File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception

reraise(exc_type, exc_value, tb)

File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus/create.sql", line 3, in top-level template code

CREATE DATABASE {{ conn|qtIdent(data.name) }}

File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__init__.py", line 1597, in qtIdent

val = str(val)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)

Isn't this about the source code encoding, and not about the input encoding?

--
Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From

Murtuza Zabuawala

Date:

12 July 2016, 14:11:38

Please refer,

http://stackoverflow.com/questions/3828723/why-should-we-not-use-sys-setdefaultencodingutf-8-in-a-py-script

Regards,

Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 4:18 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:

On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
May be because of Python2.7 but if you try in Python3. It works.

we can use PEP 0263 to fix this issue. // # -*- coding: utf-8 -*-
I tried but it didn't worked.
It gives me error:
Traceback (most recent call last):
File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 639, in msql
status, res = self.get_sql(gid, sid, data, did)
File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 686, in get_sql
SQL += self.get_new_sql(gid, sid, data, did)
File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 721, in get_new_sql
data=data, conn=self.conn
File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template
context, ctx.app)
File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 110, in _render
rv = template.render(context)
File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus/create.sql", line 3, in top-level template code
CREATE DATABASE {{ conn|qtIdent(data.name) }}
File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__init__.py", line 1597, in qtIdent
val = str(val)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)

Isn't this about the source code encoding, and not about the input encoding?

--
Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

http://www.linkedin.com/in/asheshvashi

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is reproducible to me in web application.

Now sure but the issue is when runtime(QtWebkit) sends data it does not encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

What is happening in code?
Database is created with Chinese characters successfully with ascii values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data and throws UnicodeError.

Solution:
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and Chinese characters. It works.

Also referenced the code to set default encoding to 'uff-8' before start of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers