Thread: Relaxing NaN/Infinity restriction in JSON fields

Relaxing NaN/Infinity restriction in JSON fields

From
Mitar
Date:
Hi!

When migrating from MongoDB to PostgreSQL one thing which just
surprised me now is that I cannot store NaN/Infinity in JSON fields. I
know that standard JSON restricts those values, but they are a very
common (and welcome) relaxation. What are prospects of this
restriction being lifted? It is really sad that one cannot stores
directly all IEEE 754 double precision floating point values.


Mitar

-- 
http://mitar.tnode.com/
https://twitter.com/mitar_m



Re: Relaxing NaN/Infinity restriction in JSON fields

From
Tom Lane
Date:
Mitar <mmitar@gmail.com> writes:
> When migrating from MongoDB to PostgreSQL one thing which just
> surprised me now is that I cannot store NaN/Infinity in JSON fields. I
> know that standard JSON restricts those values, but they are a very
> common (and welcome) relaxation. What are prospects of this
> restriction being lifted?

The JSON RFC is pretty clear on this matter [1]:

   Numeric values that cannot be represented in the grammar below (such
   as Infinity and NaN) are not permitted.

Getting us to deviate from the RFC so blatantly would be a very hard sell.
A large part of the point of the JSON datatype is to be interoperable;
once you give that up you may as well use some not-standard-at-all
representation.

> It is really sad that one cannot stores
> directly all IEEE 754 double precision floating point values.

There is not, and never has been, any claim that JSON numbers correspond
to the IEEE spec.

            regards, tom lane

[1] https://tools.ietf.org/html/rfc7159#page-6



Re: Relaxing NaN/Infinity restriction in JSON fields

From
Mitar
Date:
Hi!

On Mon, May 6, 2019 at 1:21 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Getting us to deviate from the RFC so blatantly would be a very hard sell.
> A large part of the point of the JSON datatype is to be interoperable;
> once you give that up you may as well use some not-standard-at-all
> representation.

Python supports that, enabled by default:

https://docs.python.org/3/library/json.html#infinite-and-nan-number-values

> There is not, and never has been, any claim that JSON numbers correspond
> to the IEEE spec.

There is note [1], but yes, it does not claim that nor I claimed that.
I am just saying that the reality is that most people these days use
IEEE spec floating numbers so it is sad that those cannot be easily
stored in JSON, or a database.


Mitar

[1] https://tools.ietf.org/html/rfc7159#page-7

-- 
http://mitar.tnode.com/
https://twitter.com/mitar_m



Re: Relaxing NaN/Infinity restriction in JSON fields

From
Alvaro Herrera
Date:
On 2019-May-07, Mitar wrote:

> On Mon, May 6, 2019 at 1:21 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> > There is not, and never has been, any claim that JSON numbers correspond
> > to the IEEE spec.
> 
> There is note [1], but yes, it does not claim that nor I claimed that.
> I am just saying that the reality is that most people these days use
> IEEE spec floating numbers so it is sad that those cannot be easily
> stored in JSON, or a database.

If you want to complain about JSON, it's IETF that you need to talk
about, not us -- we're just implementing their spec.  As for storing the
numbers in a database, you can already do that, just not on the JSON
datatype.

There is a lot of talk in the json mailing list about subnormals and why
they don't want them valid in JSON because of interoperability, and that
discussion led to the wording present in RFC7159; strangely, the wording
there about Inf/NaN predates that discussion (it's already there in
RFC4627) and I couldn't find the rationale for they being disallowed.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: Relaxing NaN/Infinity restriction in JSON fields

From
Mitar
Date:
Hi!

On Wed, May 8, 2019 at 6:09 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> If you want to complain about JSON, it's IETF that you need to talk
> about, not us -- we're just implementing their spec.  As for storing the
> numbers in a database, you can already do that, just not on the JSON
> datatype.

Yes, I see why then so many implement variations on JSON, like BSON
and stuff. So that they can have mostly compatible structure, but with
all floats and datetime structure.

What are thoughts of adding something like that? PgJSON variant. :-)


Mitar

-- 
http://mitar.tnode.com/
https://twitter.com/mitar_m