Thread: JSON serialization in pgAdmin 4
Hi Emanuele,
On Sat, May 28, 2016 at 7:59 PM, Lele Gaifax <lele@metapensiero.it> wrote:
Hi all,
first of all, let me congratulate with you for how good pgAdmin 4 is quickly
becoming. I tried it now and then, and I'm always surprised by the amount of
effort you spend on it, thank you!
Thanks!
I'd like to know if there is already a plan to properly handle date/timestamp
fields, that currently fail with an error because simplejson by default does
not know how the serialize (or deserialize) those data types.
Thanks for sharing the information.
Can you please share some more detailed information about it, where/in which condition it does fail?
One minimalistic approach would be subclassing the JSONEncoder and overriding
its default() method. This would solve the serialization problem, but I guess
that sooner or later we may want to support also the other way around.
Yeah - that's what we've done at one place.
One further step in that direction could be using nssjson[1] (my fork of
simplejson that supports date/time/datetime/UUID out of the box). I
reimplemented the same functionality on top of python-rapidjson, but the
author didn't accept my PR[2] yet. Rapidjson is much faster on most
operations, but it's also very young...
We would certainly like to take a look at it.
Thanks for sharing the information.
--
Thanks & Regards,
Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
Thanks in advance,
ciao, lele.
[1] https://pypi.python.org/pypi/nssjson
[2] https://github.com/kenrobbins/python-rapidjson/pull/35
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it | -- Fortunato Depero, 1929.
--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers
On Sat, May 28, 2016 at 11:08 PM, Lele Gaifax <lele@metapensiero.it> wrote:
Ashesh Vashi <ashesh.vashi@enterprisedb.com> writes:
> On Sat, May 28, 2016 at 7:59 PM, Lele Gaifax <lele@metapensiero.it> wrote:
>> I'd like to know if there is already a plan to properly handle
>> date/timestamp
>> fields, that currently fail with an error because simplejson by default
>> does
>> not know how the serialize (or deserialize) those data types.
>
> Can you please share some more detailed information about it, where/in
> which condition it does fail?
Consider the following simple table:
testdb=# \d users
Table "public.users"
Column | Type | Modifiers
-----------+--------------------+------------------------
id | integer | not null
name | character varying |
fullname | character varying |
birthdate | date | not null
testdb=# select * from users;
id | name | fullname | birthdate
----+-------+----------------+------------
1 | jack | Jack Jones | 1987-01-02
2 | wendy | Wendy Williams | 1978-01-02
(2 rows)
Then, executing the "View first 100 rows" action, I get
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500
Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server
encountered an internal error and was unable to complete your
request. Either the server is overloaded or there is an error in the
application.</p>
with the following traceback (slightly edited for brevity)
2016-05-28 19:15:52,824: ERROR pgadmin: Exception on /sqleditor/poll/7927153 [GET]
Traceback (most recent call last):
...
File ".../pgadmin4/web/pgadmin/utils/ajax.py", line 41, in make_json_response
response=json.dumps(doc, cls=DataTypeJSONEncoder),
File ".../python3.6/site-packages/simplejson/__init__.py", line 386, in dumps
**kw).encode(obj)
File ".../python3.6/site-packages/simplejson/encoder.py", line 269, in encode
chunks = self.iterencode(o, _one_shot=True)
File ".../python3.6/site-packages/simplejson/encoder.py", line 348, in iterencode
return _iterencode(o, 0)
File ".../pgadmin4/web/pgadmin/utils/ajax.py", line 26, in default
return json.JSONEncoder.default(self, obj)
File ".../python3.6/site-packages/simplejson/encoder.py", line 246, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.date(1987, 1, 2) is not JSON serializable
As said, the DataTypeJSONEncoder could be extended to handle dates and times
as well as datetimes and Decimals. BTW, the latter are already supported by
simplejson:
>>> import simplejson
>>> import decimal
>>> d = decimal.Decimal('3.14159')
>>> simplejson.dumps([d])
'[3.14159]'
OTOH, the following is with nssjson:
Python 2.7.11+ (default, May 9 2016, 15:54:33)
[GCC 5.3.1 20160429] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime, uuid
>>> import nssjson
>>> now = datetime.datetime.now()
>>> today = datetime.date.today()
>>> time = now.time()
>>> uid = uuid.uuid1()
>>> nssjson.dumps([uid, now, today, time], iso_datetime=True, handle_uuid=True)
'["2d628922-24f9-11e6-b746-3085a99ccac7", "2016-05-28T19:25:26.109827", "2016-05-28", "19:25:26.109827"]'
>>> nssjson.loads(_, iso_datetime=True, handle_uuid=True)
[UUID('2d628922-24f9-11e6-b746-3085a99ccac7'), datetime.datetime(2016, 5, 28, 19, 25, 26, 109827), datetime.date(2016, 5, 28), datetime.time(19, 25, 26, 109827)]
possibly using its C accelerator to serialize/deserialize those types.
Thanks for the information.
Akshay,
Please take a look at it, and see what we can do about it.
ciao, lele.--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it | -- Fortunato Depero, 1929.
--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers