Thread: SQLJSON

SQLJSON

From
Dave Cramer
Date:
I'm looking for comments on how to implement a SQLJSON type in JDBC.

As there is no getSQLJSON in the resultset interface this could only be used in getObject.



Dave Cramer

Re: SQLJSON

From
"Markus KARG"
Date:

I think it would be pretty straigtforward if SQLJSON would have the exact same interface as SQLXML. The sole difference I see is that in contrast to supporting JAXP it has to support JSONP (https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/index.html).

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Freitag, 26. Juni 2015 16:57
To: List
Subject: [JDBC] SQLJSON

 

I'm looking for comments on how to implement a SQLJSON type in JDBC.

 

As there is no getSQLJSON in the resultset interface this could only be used in getObject.

 

 


Dave Cramer

Re: SQLJSON

From
Steven Schlansker
Date:
On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:

> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>
> As there is no getSQLJSON in the resultset interface this could only be used in getObject.
>
> Notionally it would model itself after SQLXML.
https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML.html

I used JSON extensively in one of my projects, but have never used SQLXML.  I'm having trouble understanding why the
SQLXMLinterface adds any value to passing rs.getBinaryStream to your favorite JSON parser.  Especially since you would
haveto use getObject, I am not seeing how: 

rs.getObject("field", SQLJSON.class).mapToType(MyType.class)

is simpler than:
jacksonObjectMapper.readValue(rs.getBinaryStream("field"), MyType.class)

which already works today as far as I understand.  Doubly so since nobody will agree on which JSON parsing library to
use.

I'm sure I'm missing something?



Re: SQLJSON

From
Dave Cramer
Date:


Dave Cramer

On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com> wrote:

On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:

> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>
> As there is no getSQLJSON in the resultset interface this could only be used in getObject.
>
> Notionally it would model itself after SQLXML. https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML.html

I used JSON extensively in one of my projects, but have never used SQLXML.  I'm having trouble understanding why the SQLXML interface adds any value to passing rs.getBinaryStream to your favorite JSON parser.  Especially since you would have to use getObject, I am not seeing how:

rs.getObject("field", SQLJSON.class).mapToType(MyType.class)

is simpler than:
jacksonObjectMapper.readValue(rs.getBinaryStream("field"), MyType.class)

which already works today as far as I understand.  Doubly so since nobody will agree on which JSON parsing library to use.

I'm sure I'm missing something?


I don't think you are; as you rightly pointed out now we would have to add a json parser to the driver, which I'm reluctant to do 

Re: SQLJSON

From
Steven Schlansker
Date:
On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com> wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be used in getObject.
> >
> > Notionally it would model itself after SQLXML.
https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML.html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.  I'm having trouble understanding why the
SQLXMLinterface adds any value to passing rs.getBinaryStream to your favorite JSON parser.  Especially since you would
haveto use getObject, I am not seeing how: 
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"), MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody will agree on which JSON parsing library to
use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to add a json parser to the driver, which I'm
reluctantto do  
>

If this feature is developed, I think the JSON parser should be pluggable and optional if possible.  Then users that do
notwant it do not need to drag in a large dependency. 

That said, without a more convincing use case or a compelling API that we could easily add, I don't see this interface
being"worth its weight" as an addition. 



Re: SQLJSON

From
"Markus KARG"
Date:
Please read this regarding pluggability:
https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/javax/
json/spi/JsonProvider.html.

Please read this regarding inability to agree on a standard:
https://jcp.org/en/jsr/results?id=5486.

The merits of adding a SQLJSON type to JDBC would be that in case an RDBMS
might have some kind of special support for high-performance object-tree
transmission in the network protocol (like lazy lookup and delayed
transmission of child objects for example, or using binary transmission
instead of JSONP plain-text), the driver could make use of that knowledge
and work in the fastest and memory-saving way. That benefits are impossible
to gain when using a character stream, as that requires to explicitly
inflate to a full JSONP representation in-memory just to parse that string
in turn in a second step.

whether or not to support JSON is not up to the implementors of drivers, but
up to the JDBC specification vendor. I doubt that with JSONP being a
mandatory part of Java EE they will leave room for not providing JSONP in
JDBC. We might be reluctant for now, but I believe that JDBC5 will enforce
it.

Regards
-Markus

-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Steven Schlansker
Sent: Freitag, 26. Juni 2015 19:29
To: Dave Cramer
Cc: List
Subject: Re: [JDBC] SQLJSON


On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
> >
> > Notionally it would model itself after SQLXML.
> > https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
> > .html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.
I'm having trouble understanding why the SQLXML interface adds any value to
passing rs.getBinaryStream to your favorite JSON parser.  Especially since
you would have to use getObject, I am not seeing how:
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
> MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to
> add a json parser to the driver, which I'm reluctant to do
>

If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.

That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes
to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Dave Cramer
Date:
Well I said "currently possible". The architects of JSON need to realize that we need a binary transfer mechanism.

As for lazy streaming that would require a new protocol.

also discussion is good if for no other reason to stimulate ideas

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 08:52, Markus KARG <markus@headcrashing.eu> wrote:

Then why starting a discussion about SQLJSON? ;-)

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Samstag, 27. Juni 2015 13:12
To: Markus KARG
Subject: Re: [JDBC] SQLJSON

 

 

On 27 June 2015 at 03:12, Markus KARG <markus@headcrashing.eu> wrote:

Please read this regarding pluggability:
https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/javax/
json/spi/JsonProvider.html
.

Please read this regarding inability to agree on a standard:
https://jcp.org/en/jsr/results?id=5486.

The merits of adding a SQLJSON type to JDBC would be that in case an RDBMS
might have some kind of special support for high-performance object-tree
transmission in the network protocol (like lazy lookup and delayed
transmission of child objects for example, or using binary transmission
instead of JSONP plain-text), the driver could make use of that knowledge
and work in the fastest and memory-saving way. That benefits are impossible
to gain when using a character stream, as that requires to explicitly
inflate to a full JSONP representation in-memory just to parse that string
in turn in a second step.

Unfortunately none of the above is currently possible in PostgreSQL

 



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Steven Schlansker
Sent: Freitag, 26. Juni 2015 19:29
To: Dave Cramer
Cc: List
Subject: Re: [JDBC] SQLJSON


On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
> >
> > Notionally it would model itself after SQLXML.
> > https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
> > .html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.
I'm having trouble understanding why the SQLXML interface adds any value to
passing rs.getBinaryStream to your favorite JSON parser.  Especially since
you would have to use getObject, I am not seeing how:
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
> MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to
> add a json parser to the driver, which I'm reluctant to do
>

If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.

That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes
to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 


Re: SQLJSON

From
Sehrope Sarkuni
Date:
Is there an actual SQLJSON type or in a spec somewhere? I can't find any info about it anywhere.

Or are we just using SQLJSON as a term to refer to a non-String representation of JSON coming back from a ResultSet?

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Re: SQLJSON

From
Christopher BROWN
Date:
Hello,

I've enquired about JSON support in the driver previously:

Ideally, if the PostgreSQL database protocol can stream JSON data (binary or text), then any solution other than current "read it all into memory as a big fat string or array" would be good (i.e.: being able to read using "The Streaming API", as described here -- http://www.oracle.com/technetwork/articles/java/json-1973242.html -- in addition to "The Object Model API").  The object model API is a convenience, but the streaming API allows applications to avoid memory usage spikes, retaining only what is needed and discarding the rest, with performance, memory, and garbage-collection benefits.

If the driver is compiled against the JSONP interfaces, as an optional dependency (for normal classloading, OSGi classloading, and Java9 modular classloading), then it carries no extra "weight" if not used.  The JSONP dependency should not be bundled with the driver, it should be made available to it by the application; the application would need it anyway before being able to process such interfaces via the driver anyway.  Similarly, the JSONP API uses the service loader API to load implementations, however, as an OSGi user, I'd be pleased if there was an alternative (because the service loader doesn't help you hot-swap implementations at runtime without restarting the application.  I'd personally appreciate it if the driver defined an interface for retrieving custom implementations of all the JSONP xxxFactory interfaces, delegating by default to the service loader-based "Json" class unless overridden (I'd override it, others may be happy with default behavior).

Hope that helps,
Christopher



On 27 June 2015 at 15:01, Dave Cramer <pg@fastcrypt.com> wrote:
Well I said "currently possible". The architects of JSON need to realize that we need a binary transfer mechanism.

As for lazy streaming that would require a new protocol.

also discussion is good if for no other reason to stimulate ideas

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 08:52, Markus KARG <markus@headcrashing.eu> wrote:

Then why starting a discussion about SQLJSON? ;-)

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Samstag, 27. Juni 2015 13:12
To: Markus KARG
Subject: Re: [JDBC] SQLJSON

 

 

On 27 June 2015 at 03:12, Markus KARG <markus@headcrashing.eu> wrote:

Please read this regarding pluggability:
https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/javax/
json/spi/JsonProvider.html
.

Please read this regarding inability to agree on a standard:
https://jcp.org/en/jsr/results?id=5486.

The merits of adding a SQLJSON type to JDBC would be that in case an RDBMS
might have some kind of special support for high-performance object-tree
transmission in the network protocol (like lazy lookup and delayed
transmission of child objects for example, or using binary transmission
instead of JSONP plain-text), the driver could make use of that knowledge
and work in the fastest and memory-saving way. That benefits are impossible
to gain when using a character stream, as that requires to explicitly
inflate to a full JSONP representation in-memory just to parse that string
in turn in a second step.

Unfortunately none of the above is currently possible in PostgreSQL

 



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Steven Schlansker
Sent: Freitag, 26. Juni 2015 19:29
To: Dave Cramer
Cc: List
Subject: Re: [JDBC] SQLJSON


On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
> >
> > Notionally it would model itself after SQLXML.
> > https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
> > .html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.
I'm having trouble understanding why the SQLXML interface adds any value to
passing rs.getBinaryStream to your favorite JSON parser.  Especially since
you would have to use getObject, I am not seeing how:
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
> MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to
> add a json parser to the driver, which I'm reluctant to do
>

If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.

That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes
to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 



Re: SQLJSON

From
Dave Cramer
Date:
Serhope,

No there is no SQLJSON type.... seems the java community can't even agree on a parser. Everyone else has decided boon https://github.com/boonproject/boon is the way to go.


Christopher, resurrecting your thread; The code below seems doable though 

PreparedStatement stmt = connection.prepareStatement("UPDATE MYTABLE SET
MYJSONCOL = ? WHERE MYID = ?");
stmt.setObject(1, map);
stmt.setInt(2, 123);


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 09:23, Christopher BROWN <brown@reflexe.fr> wrote:
Hello,

I've enquired about JSON support in the driver previously:

Ideally, if the PostgreSQL database protocol can stream JSON data (binary or text), then any solution other than current "read it all into memory as a big fat string or array" would be good (i.e.: being able to read using "The Streaming API", as described here -- http://www.oracle.com/technetwork/articles/java/json-1973242.html -- in addition to "The Object Model API").  The object model API is a convenience, but the streaming API allows applications to avoid memory usage spikes, retaining only what is needed and discarding the rest, with performance, memory, and garbage-collection benefits.

If the driver is compiled against the JSONP interfaces, as an optional dependency (for normal classloading, OSGi classloading, and Java9 modular classloading), then it carries no extra "weight" if not used.  The JSONP dependency should not be bundled with the driver, it should be made available to it by the application; the application would need it anyway before being able to process such interfaces via the driver anyway.  Similarly, the JSONP API uses the service loader API to load implementations, however, as an OSGi user, I'd be pleased if there was an alternative (because the service loader doesn't help you hot-swap implementations at runtime without restarting the application.  I'd personally appreciate it if the driver defined an interface for retrieving custom implementations of all the JSONP xxxFactory interfaces, delegating by default to the service loader-based "Json" class unless overridden (I'd override it, others may be happy with default behavior).

Hope that helps,
Christopher



On 27 June 2015 at 15:01, Dave Cramer <pg@fastcrypt.com> wrote:
Well I said "currently possible". The architects of JSON need to realize that we need a binary transfer mechanism.

As for lazy streaming that would require a new protocol.

also discussion is good if for no other reason to stimulate ideas

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 08:52, Markus KARG <markus@headcrashing.eu> wrote:

Then why starting a discussion about SQLJSON? ;-)

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Samstag, 27. Juni 2015 13:12
To: Markus KARG
Subject: Re: [JDBC] SQLJSON

 

 

On 27 June 2015 at 03:12, Markus KARG <markus@headcrashing.eu> wrote:

Please read this regarding pluggability:
https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/javax/
json/spi/JsonProvider.html
.

Please read this regarding inability to agree on a standard:
https://jcp.org/en/jsr/results?id=5486.

The merits of adding a SQLJSON type to JDBC would be that in case an RDBMS
might have some kind of special support for high-performance object-tree
transmission in the network protocol (like lazy lookup and delayed
transmission of child objects for example, or using binary transmission
instead of JSONP plain-text), the driver could make use of that knowledge
and work in the fastest and memory-saving way. That benefits are impossible
to gain when using a character stream, as that requires to explicitly
inflate to a full JSONP representation in-memory just to parse that string
in turn in a second step.

Unfortunately none of the above is currently possible in PostgreSQL

 



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Steven Schlansker
Sent: Freitag, 26. Juni 2015 19:29
To: Dave Cramer
Cc: List
Subject: Re: [JDBC] SQLJSON


On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
> >
> > Notionally it would model itself after SQLXML.
> > https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
> > .html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.
I'm having trouble understanding why the SQLXML interface adds any value to
passing rs.getBinaryStream to your favorite JSON parser.  Especially since
you would have to use getObject, I am not seeing how:
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
> MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to
> add a json parser to the driver, which I'm reluctant to do
>

If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.

That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes
to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 




Re: SQLJSON

From
Christopher BROWN
Date:
There was a discussion about Boon here:

In short, it may achieve impressive results due to biased benchmarks, and in many cases just reads in a JSON document as a big fat string and defers real deserialization.  It also apparently useds "com.sun" classes.  I'm not sure it's the way to go based on that information, it would need to be checked out a bit more.

With regards to storing "maps", wasn't there an issue of conflict with HSTORE ?

--
Christopher


On 27 June 2015 at 15:34, Dave Cramer <pg@fastcrypt.com> wrote:
Serhope,

No there is no SQLJSON type.... seems the java community can't even agree on a parser. Everyone else has decided boon https://github.com/boonproject/boon is the way to go.


Christopher, resurrecting your thread; The code below seems doable though 

PreparedStatement stmt = connection.prepareStatement("UPDATE MYTABLE SET
MYJSONCOL = ? WHERE MYID = ?");
stmt.setObject(1, map);
stmt.setInt(2, 123);


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 09:23, Christopher BROWN <brown@reflexe.fr> wrote:
Hello,

I've enquired about JSON support in the driver previously:

Ideally, if the PostgreSQL database protocol can stream JSON data (binary or text), then any solution other than current "read it all into memory as a big fat string or array" would be good (i.e.: being able to read using "The Streaming API", as described here -- http://www.oracle.com/technetwork/articles/java/json-1973242.html -- in addition to "The Object Model API").  The object model API is a convenience, but the streaming API allows applications to avoid memory usage spikes, retaining only what is needed and discarding the rest, with performance, memory, and garbage-collection benefits.

If the driver is compiled against the JSONP interfaces, as an optional dependency (for normal classloading, OSGi classloading, and Java9 modular classloading), then it carries no extra "weight" if not used.  The JSONP dependency should not be bundled with the driver, it should be made available to it by the application; the application would need it anyway before being able to process such interfaces via the driver anyway.  Similarly, the JSONP API uses the service loader API to load implementations, however, as an OSGi user, I'd be pleased if there was an alternative (because the service loader doesn't help you hot-swap implementations at runtime without restarting the application.  I'd personally appreciate it if the driver defined an interface for retrieving custom implementations of all the JSONP xxxFactory interfaces, delegating by default to the service loader-based "Json" class unless overridden (I'd override it, others may be happy with default behavior).

Hope that helps,
Christopher



On 27 June 2015 at 15:01, Dave Cramer <pg@fastcrypt.com> wrote:
Well I said "currently possible". The architects of JSON need to realize that we need a binary transfer mechanism.

As for lazy streaming that would require a new protocol.

also discussion is good if for no other reason to stimulate ideas

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 08:52, Markus KARG <markus@headcrashing.eu> wrote:

Then why starting a discussion about SQLJSON? ;-)

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Samstag, 27. Juni 2015 13:12
To: Markus KARG
Subject: Re: [JDBC] SQLJSON

 

 

On 27 June 2015 at 03:12, Markus KARG <markus@headcrashing.eu> wrote:

Please read this regarding pluggability:
https://json-processing-spec.java.net/nonav/releases/1.0/fcs/javadocs/javax/
json/spi/JsonProvider.html
.

Please read this regarding inability to agree on a standard:
https://jcp.org/en/jsr/results?id=5486.

The merits of adding a SQLJSON type to JDBC would be that in case an RDBMS
might have some kind of special support for high-performance object-tree
transmission in the network protocol (like lazy lookup and delayed
transmission of child objects for example, or using binary transmission
instead of JSONP plain-text), the driver could make use of that knowledge
and work in the fastest and memory-saving way. That benefits are impossible
to gain when using a character stream, as that requires to explicitly
inflate to a full JSONP representation in-memory just to parse that string
in turn in a second step.

Unfortunately none of the above is currently possible in PostgreSQL

 



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Steven Schlansker
Sent: Freitag, 26. Juni 2015 19:29
To: Dave Cramer
Cc: List
Subject: Re: [JDBC] SQLJSON


On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:

>
> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>
> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
> > I'm looking for comments on how to implement a SQLJSON type in JDBC.
> >
> > As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
> >
> > Notionally it would model itself after SQLXML.
> > https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
> > .html
>
> I used JSON extensively in one of my projects, but have never used SQLXML.
I'm having trouble understanding why the SQLXML interface adds any value to
passing rs.getBinaryStream to your favorite JSON parser.  Especially since
you would have to use getObject, I am not seeing how:
>
> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>
> is simpler than:
> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
> MyType.class)
>
> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>
> I'm sure I'm missing something?
>
>
> I don't think you are; as you rightly pointed out now we would have to
> add a json parser to the driver, which I'm reluctant to do
>

If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.

That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make changes
to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 





Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 26/06/15 19:29, Steven Schlansker wrote:
> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com> wrote:
>>
>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>
>>> As there is no getSQLJSON in the resultset interface this could only be used in getObject.
>>>
>>> Notionally it would model itself after SQLXML.
https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML.html
>> I used JSON extensively in one of my projects, but have never used SQLXML.  I'm having trouble understanding why the
SQLXMLinterface adds any value to passing rs.getBinaryStream to your favorite JSON parser.  Especially since you would
haveto use getObject, I am not seeing how: 
>>
>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>
>> is simpler than:
>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"), MyType.class)
>>
>> which already works today as far as I understand.  Doubly so since nobody will agree on which JSON parsing library
touse. 
>>
>> I'm sure I'm missing something?
>>
>>
>> I don't think you are; as you rightly pointed out now we would have to add a json parser to the driver, which I'm
reluctantto do 
>>
> If this feature is developed, I think the JSON parser should be pluggable and optional if possible.  Then users that
donot want it do not need to drag in a large dependency. 
>
> That said, without a more convincing use case or a compelling API that we could easily add, I don't see this
interfacebeing "worth its weight" as an addition. 
>
>
>

     Hi List.

     I always try to think more from the user perspective than from the
developer one. This is also going to be the case.

     Having JSON support in the PostgreSQL JDBC driver is a *must*.
jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support
in one of the most used PostgreSQL drivers. No blame here (at all), just
trying to support my point here.

     I don't see the advantage of using SQLJSON, although I wouldn't
argue against. What I clearly believe is that at the end of the day you
should be able to easily return a javax.json.JsonObject from a
ResultSet. Being a JavaEE standard and a JSR, I believe it's the best
(and obvious) choice.

     Regarding pluggability, JSR353's SPI mechanism is good, but asking
the user to provide a further dependency "just for reading JSON" seems
again to me not good from the user perspective. I'd ideally expect json
to be supported as-is, as with any other datatype. Having the SPI we
could choose whatever implementation we want. Is that enlarging the
driver's size? So what? Users want easy-of-use, not driver size. And
there are many mechanisms to reduce size for unused classes.

     If any, my 2 cents are: let's add JSON, let's take JSR353 as an API
for it and let's make it as easy as possible for final users to use it.

     Regards,

     Álvaro



--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
"Markus KARG"
Date:
Álvaro,

I also think we should look at the topic with the eyes of the user, but I
have to disagree in two point exactly due to that attitude:

(1) JSON is definitively not just another datatype. Just like XML, it
provides the ability to return a complete object graph as the content of one
column of one row. Hence, it contains the same amount of data that in
pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
words, from the complexity of the information and from the aspect of end
user performance, it "feels" like a ResulSet that contains ResultSets that
contains ResultSets... Handling it just like any other datatype is what
pgjdbc does currently, and it provides very poor performance due to that, as
it simply ships the complete graph - independent of the fact whether the end
user will process is completely or just pick a single node of it. Asking to
handle JSON as just any other data type is like asking to give up ResultSet
in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
of an end user, the answer must be, to NOT handle JSON like any other
datatype, and NOT return a JsonObject, but instead provide a streaming API,
just like SQLXML does. This allows to only transfer and process that few
nodes the the end user actually likes to have, just like an end user
typically will never process a complete ResultSet but process it iteratively
and most typically filtered.

(2) The user MUST be always asked to provide a JSON parser, as it is HIM who
wants to use JSON, and so it is HIS choice to select one among those
fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc picks
and / or provides one. Most typically this will end up in the same design
choice than most XML applications ended up with: Relying on Java SE / EE
providing a default product, like it is the case with JAXB (Xerces). The
only thing pgjdbc can really do NOW is to support the JSON SPI *if* there is
a processor on the classpath, and once the postgresql server produces an
improved streaming procotol for JSON / XML we can couple the client's JSON
event handler with the server's protocol events.

Regards
Markus

-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Samstag, 27. Juni 2015 18:56
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 26/06/15 19:29, Steven Schlansker wrote:
> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>>
>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>
>>> As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
>>>
>>> Notionally it would model itself after SQLXML.
>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>> .html
>> I used JSON extensively in one of my projects, but have never used
SQLXML.  I'm having trouble understanding why the SQLXML interface adds any
value to passing rs.getBinaryStream to your favorite JSON parser.
Especially since you would have to use getObject, I am not seeing how:
>>
>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>
>> is simpler than:
>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>> MyType.class)
>>
>> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>>
>> I'm sure I'm missing something?
>>
>>
>> I don't think you are; as you rightly pointed out now we would have
>> to add a json parser to the driver, which I'm reluctant to do
>>
> If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.
>
> That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.
>
>
>

     Hi List.

     I always try to think more from the user perspective than from the
developer one. This is also going to be the case.

     Having JSON support in the PostgreSQL JDBC driver is a *must*.
jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support in
one of the most used PostgreSQL drivers. No blame here (at all), just trying
to support my point here.

     I don't see the advantage of using SQLJSON, although I wouldn't argue
against. What I clearly believe is that at the end of the day you should be
able to easily return a javax.json.JsonObject from a ResultSet. Being a
JavaEE standard and a JSR, I believe it's the best (and obvious) choice.

     Regarding pluggability, JSR353's SPI mechanism is good, but asking the
user to provide a further dependency "just for reading JSON" seems again to
me not good from the user perspective. I'd ideally expect json to be
supported as-is, as with any other datatype. Having the SPI we could choose
whatever implementation we want. Is that enlarging the driver's size? So
what? Users want easy-of-use, not driver size. And there are many mechanisms
to reduce size for unused classes.

     If any, my 2 cents are: let's add JSON, let's take JSR353 as an API for
it and let's make it as easy as possible for final users to use it.

     Regards,

     Álvaro



--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Dave Cramer
Date:


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 27 June 2015 at 14:07, Markus KARG <markus@headcrashing.eu> wrote:
Álvaro,

I also think we should look at the topic with the eyes of the user, but I
have to disagree in two point exactly due to that attitude:

(1) JSON is definitively not just another datatype. Just like XML, it
provides the ability to return a complete object graph as the content of one
column of one row. Hence, it contains the same amount of data that in
pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
words, from the complexity of the information and from the aspect of end
user performance, it "feels" like a ResulSet that contains ResultSets that
contains ResultSets... Handling it just like any other datatype is what
pgjdbc does currently, and it provides very poor performance due to that, as
it simply ships the complete graph - independent of the fact whether the end
user will process is completely or just pick a single node of it. Asking to
handle JSON as just any other data type is like asking to give up ResultSet
in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
of an end user, the answer must be, to NOT handle JSON like any other
datatype, and NOT return a JsonObject, but instead provide a streaming API,
just like SQLXML does. This allows to only transfer and process that few
nodes the the end user actually likes to have, just like an end user
typically will never process a complete ResultSet but process it iteratively
and most typically filtered.


Unfortunately as PostgreSQL always returns the entire object there is no real "Streaming" we will have to buffer the results in a String or CharArray and "stream" from there.

 
(2) The user MUST be always asked to provide a JSON parser, as it is HIM who
wants to use JSON, and so it is HIS choice to select one among those
fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc picks
and / or provides one. Most typically this will end up in the same design
choice than most XML applications ended up with: Relying on Java SE / EE
providing a default product, like it is the case with JAXB (Xerces). The
only thing pgjdbc can really do NOW is to support the JSON SPI *if* there is
a processor on the classpath, and once the postgresql server produces an
improved streaming procotol for JSON / XML we can couple the client's JSON
event handler with the server's protocol events.

Given that Java itself can'd decide on a parser (yes I'm aware of the various JSR's) we might want to think of defining our own parser api which the user can provide an implementation of. This could just be a wrapper around jackson, but would give the user more flexibility ??

Dave



-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Samstag, 27. Juni 2015 18:56
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 26/06/15 19:29, Steven Schlansker wrote:
> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>>
>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>
>>> As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
>>>
>>> Notionally it would model itself after SQLXML.
>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>> .html
>> I used JSON extensively in one of my projects, but have never used
SQLXML.  I'm having trouble understanding why the SQLXML interface adds any
value to passing rs.getBinaryStream to your favorite JSON parser.
Especially since you would have to use getObject, I am not seeing how:
>>
>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>
>> is simpler than:
>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>> MyType.class)
>>
>> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>>
>> I'm sure I'm missing something?
>>
>>
>> I don't think you are; as you rightly pointed out now we would have
>> to add a json parser to the driver, which I'm reluctant to do
>>
> If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.
>
> That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.
>
>
>

     Hi List.

     I always try to think more from the user perspective than from the
developer one. This is also going to be the case.

     Having JSON support in the PostgreSQL JDBC driver is a *must*.
jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support in
one of the most used PostgreSQL drivers. No blame here (at all), just trying
to support my point here.

     I don't see the advantage of using SQLJSON, although I wouldn't argue
against. What I clearly believe is that at the end of the day you should be
able to easily return a javax.json.JsonObject from a ResultSet. Being a
JavaEE standard and a JSR, I believe it's the best (and obvious) choice.

     Regarding pluggability, JSR353's SPI mechanism is good, but asking the
user to provide a further dependency "just for reading JSON" seems again to
me not good from the user perspective. I'd ideally expect json to be
supported as-is, as with any other datatype. Having the SPI we could choose
whatever implementation we want. Is that enlarging the driver's size? So
what? Users want easy-of-use, not driver size. And there are many mechanisms
to reduce size for unused classes.

     If any, my 2 cents are: let's add JSON, let's take JSR353 as an API for
it and let's make it as easy as possible for final users to use it.

     Regards,

     Álvaro



--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 27/06/15 20:07, Markus KARG wrote:
> Álvaro,
>
> I also think we should look at the topic with the eyes of the user, but I
> have to disagree in two point exactly due to that attitude:
     Hi Markus. Good that we agree on that :)
>
> (1) JSON is definitively not just another datatype. Just like XML, it
> provides the ability to return a complete object graph as the content of one
> column of one row. Hence, it contains the same amount of data that in
> pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
> words, from the complexity of the information and from the aspect of end
> user performance, it "feels" like a ResulSet that contains ResultSets that
> contains ResultSets... Handling it just like any other datatype is what
> pgjdbc does currently, and it provides very poor performance due to that, as
> it simply ships the complete graph - independent of the fact whether the end
> user will process is completely or just pick a single node of it. Asking to
> handle JSON as just any other data type is like asking to give up ResultSet
> in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
> of an end user, the answer must be, to NOT handle JSON like any other
> datatype, and NOT return a JsonObject, but instead provide a streaming API,
> just like SQLXML does. This allows to only transfer and process that few
> nodes the the end user actually likes to have, just like an end user
> typically will never process a complete ResultSet but process it iteratively
> and most typically filtered.
     When I said "as just another datatype", I mean "as simple as". So
from a user perspective I would expect to have a get...JSON() method
with return value an object similar to (or IMHO exactly) JsonObject. In
other words, something directly usable.

     As Dave Cramer correctly points out in response earlier in this
thread, there is no way in current PostgreSQL support to send a partial
chunk of the JSON, so --unfortunately-- streaming API is not possible.
Not being it possible, why not provide methods to return JsonObject, the
Java EE standard and defined by a JSR?

>
> (2) The user MUST be always asked to provide a JSON parser, as it is HIM who
> wants to use JSON, and so it is HIS choice to select one among those
> fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc picks
> and / or provides one. Most typically this will end up in the same design
> choice than most XML applications ended up with: Relying on Java SE / EE
> providing a default product, like it is the case with JAXB (Xerces). The
> only thing pgjdbc can really do NOW is to support the JSON SPI *if* there is
> a processor on the classpath, and once the postgresql server produces an
> improved streaming procotol for JSON / XML we can couple the client's JSON
> event handler with the server's protocol events.

     Why not instead provide a default parser and let the user choose
another one if so she wants? It makes life easier for 90% of users who
don't care or even know about which parser to choose. Providing a
sensible default (or just a working default) is better than providing none.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
> Regards
> Markus
>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Samstag, 27. Juni 2015 18:56
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
> On 26/06/15 19:29, Steven Schlansker wrote:
>> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
> wrote:
>>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>>
>>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>>
>>>> As there is no getSQLJSON in the resultset interface this could only be
> used in getObject.
>>>> Notionally it would model itself after SQLXML.
>>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>>> .html
>>> I used JSON extensively in one of my projects, but have never used
> SQLXML.  I'm having trouble understanding why the SQLXML interface adds any
> value to passing rs.getBinaryStream to your favorite JSON parser.
> Especially since you would have to use getObject, I am not seeing how:
>>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>>
>>> is simpler than:
>>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>>> MyType.class)
>>>
>>> which already works today as far as I understand.  Doubly so since nobody
> will agree on which JSON parsing library to use.
>>> I'm sure I'm missing something?
>>>
>>>
>>> I don't think you are; as you rightly pointed out now we would have
>>> to add a json parser to the driver, which I'm reluctant to do
>>>
>> If this feature is developed, I think the JSON parser should be pluggable
> and optional if possible.  Then users that do not want it do not need to
> drag in a large dependency.
>> That said, without a more convincing use case or a compelling API that we
> could easily add, I don't see this interface being "worth its weight" as an
> addition.
>>
>>
>       Hi List.
>
>       I always try to think more from the user perspective than from the
> developer one. This is also going to be the case.
>
>       Having JSON support in the PostgreSQL JDBC driver is a *must*.
> jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support in
> one of the most used PostgreSQL drivers. No blame here (at all), just trying
> to support my point here.
>
>       I don't see the advantage of using SQLJSON, although I wouldn't argue
> against. What I clearly believe is that at the end of the day you should be
> able to easily return a javax.json.JsonObject from a ResultSet. Being a
> JavaEE standard and a JSR, I believe it's the best (and obvious) choice.
>
>       Regarding pluggability, JSR353's SPI mechanism is good, but asking the
> user to provide a further dependency "just for reading JSON" seems again to
> me not good from the user perspective. I'd ideally expect json to be
> supported as-is, as with any other datatype. Having the SPI we could choose
> whatever implementation we want. Is that enlarging the driver's size? So
> what? Users want easy-of-use, not driver size. And there are many mechanisms
> to reduce size for unused classes.
>
>       If any, my 2 cents are: let's add JSON, let's take JSR353 as an API for
> it and let's make it as easy as possible for final users to use it.
>
>       Regards,
>
>       Álvaro
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 27/06/15 22:10, Dave Cramer wrote:



 
(2) The user MUST be always asked to provide a JSON parser, as it is HIM who
wants to use JSON, and so it is HIS choice to select one among those
fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc picks
and / or provides one. Most typically this will end up in the same design
choice than most XML applications ended up with: Relying on Java SE / EE
providing a default product, like it is the case with JAXB (Xerces). The
only thing pgjdbc can really do NOW is to support the JSON SPI *if* there is
a processor on the classpath, and once the postgresql server produces an
improved streaming procotol for JSON / XML we can couple the client's JSON
event handler with the server's protocol events.

Given that Java itself can'd decide on a parser (yes I'm aware of the various JSR's) we might want to think of defining our own parser api which the user can provide an implementation of. This could just be a wrapper around jackson, but would give the user more flexibility ??

    Hi Dave.

    I agree that there are several JSON libraries, and there seems to be no consensus on the "best" one. However, there is consensus on the API, and that's clearly JSR353, which is present in JavaEE7. And has a SPI to plug implementations, so I see no strong reason to have the API based on that. Creating our own parser API seems to be a little bit re-inventing the wheel. I feel there are better uses to our developer bandwidth :)

    Best regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave



-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Samstag, 27. Juni 2015 18:56
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 26/06/15 19:29, Steven Schlansker wrote:
> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>>
>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>
>>> As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
>>>
>>> Notionally it would model itself after SQLXML.
>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>> .html
>> I used JSON extensively in one of my projects, but have never used
SQLXML.  I'm having trouble understanding why the SQLXML interface adds any
value to passing rs.getBinaryStream to your favorite JSON parser.
Especially since you would have to use getObject, I am not seeing how:
>>
>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>
>> is simpler than:
>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>> MyType.class)
>>
>> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>>
>> I'm sure I'm missing something?
>>
>>
>> I don't think you are; as you rightly pointed out now we would have
>> to add a json parser to the driver, which I'm reluctant to do
>>
> If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.
>
> That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.
>
>
>

     Hi List.

     I always try to think more from the user perspective than from the
developer one. This is also going to be the case.

     Having JSON support in the PostgreSQL JDBC driver is a *must*.
jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support in
one of the most used PostgreSQL drivers. No blame here (at all), just trying
to support my point here.

     I don't see the advantage of using SQLJSON, although I wouldn't argue
against. What I clearly believe is that at the end of the day you should be
able to easily return a javax.json.JsonObject from a ResultSet. Being a
JavaEE standard and a JSR, I believe it's the best (and obvious) choice.

     Regarding pluggability, JSR353's SPI mechanism is good, but asking the
user to provide a further dependency "just for reading JSON" seems again to
me not good from the user perspective. I'd ideally expect json to be
supported as-is, as with any other datatype. Having the SPI we could choose
whatever implementation we want. Is that enlarging the driver's size? So
what? Users want easy-of-use, not driver size. And there are many mechanisms
to reduce size for unused classes.

     If any, my 2 cents are: let's add JSON, let's take JSR353 as an API for
it and let's make it as easy as possible for final users to use it.

     Regards,

     Álvaro



--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc


Re: SQLJSON

From
"Markus KARG"
Date:

There is no need for our own parser API. There is one official standard that defines THE parser API for JSON, which is JSONP SPI. No need to reinvent the wheel. I do not see what you mean with "Given that Java itself can't decide" as the JCP DID decide. THEY and nobody else decides what "Java" is.

 

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Samstag, 27. Juni 2015 22:10
To: Markus KARG
Cc: List
Subject: Re: [JDBC] SQLJSON

 

 


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 27 June 2015 at 14:07, Markus KARG <markus@headcrashing.eu> wrote:

Álvaro,

I also think we should look at the topic with the eyes of the user, but I
have to disagree in two point exactly due to that attitude:

(1) JSON is definitively not just another datatype. Just like XML, it
provides the ability to return a complete object graph as the content of one
column of one row. Hence, it contains the same amount of data that in
pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
words, from the complexity of the information and from the aspect of end
user performance, it "feels" like a ResulSet that contains ResultSets that
contains ResultSets... Handling it just like any other datatype is what
pgjdbc does currently, and it provides very poor performance due to that, as
it simply ships the complete graph - independent of the fact whether the end
user will process is completely or just pick a single node of it. Asking to
handle JSON as just any other data type is like asking to give up ResultSet
in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
of an end user, the answer must be, to NOT handle JSON like any other
datatype, and NOT return a JsonObject, but instead provide a streaming API,
just like SQLXML does. This allows to only transfer and process that few
nodes the the end user actually likes to have, just like an end user
typically will never process a complete ResultSet but process it iteratively
and most typically filtered.

 

Unfortunately as PostgreSQL always returns the entire object there is no real "Streaming" we will have to buffer the results in a String or CharArray and "stream" from there.

 

 

(2) The user MUST be always asked to provide a JSON parser, as it is HIM who
wants to use JSON, and so it is HIS choice to select one among those
fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc picks
and / or provides one. Most typically this will end up in the same design
choice than most XML applications ended up with: Relying on Java SE / EE
providing a default product, like it is the case with JAXB (Xerces). The
only thing pgjdbc can really do NOW is to support the JSON SPI *if* there is
a processor on the classpath, and once the postgresql server produces an
improved streaming procotol for JSON / XML we can couple the client's JSON
event handler with the server's protocol events.

 

Given that Java itself can'd decide on a parser (yes I'm aware of the various JSR's) we might want to think of defining our own parser api which the user can provide an implementation of. This could just be a wrapper around jackson, but would give the user more flexibility ??

 

Dave

 

 


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Samstag, 27. Juni 2015 18:56
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 26/06/15 19:29, Steven Schlansker wrote:
> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>
>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
wrote:
>>
>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>
>>> As there is no getSQLJSON in the resultset interface this could only be
used in getObject.
>>>
>>> Notionally it would model itself after SQLXML.
>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>> .html
>> I used JSON extensively in one of my projects, but have never used
SQLXML.  I'm having trouble understanding why the SQLXML interface adds any
value to passing rs.getBinaryStream to your favorite JSON parser.
Especially since you would have to use getObject, I am not seeing how:
>>
>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>
>> is simpler than:
>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>> MyType.class)
>>
>> which already works today as far as I understand.  Doubly so since nobody
will agree on which JSON parsing library to use.
>>
>> I'm sure I'm missing something?
>>
>>
>> I don't think you are; as you rightly pointed out now we would have
>> to add a json parser to the driver, which I'm reluctant to do
>>
> If this feature is developed, I think the JSON parser should be pluggable
and optional if possible.  Then users that do not want it do not need to
drag in a large dependency.
>
> That said, without a more convincing use case or a compelling API that we
could easily add, I don't see this interface being "worth its weight" as an
addition.
>
>
>

     Hi List.

     I always try to think more from the user perspective than from the
developer one. This is also going to be the case.

     Having JSON support in the PostgreSQL JDBC driver is a *must*.
jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support in
one of the most used PostgreSQL drivers. No blame here (at all), just trying
to support my point here.

     I don't see the advantage of using SQLJSON, although I wouldn't argue
against. What I clearly believe is that at the end of the day you should be
able to easily return a javax.json.JsonObject from a ResultSet. Being a
JavaEE standard and a JSR, I believe it's the best (and obvious) choice.

     Regarding pluggability, JSR353's SPI mechanism is good, but asking the
user to provide a further dependency "just for reading JSON" seems again to
me not good from the user perspective. I'd ideally expect json to be
supported as-is, as with any other datatype. Having the SPI we could choose
whatever implementation we want. Is that enlarging the driver's size? So
what? Users want easy-of-use, not driver size. And there are many mechanisms
to reduce size for unused classes.

     If any, my 2 cents are: let's add JSON, let's take JSR353 as an API for
it and let's make it as easy as possible for final users to use it.

     Regards,

     Álvaro



--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 

Re: SQLJSON

From
"Markus KARG"
Date:
(1) I have no problem with providing a JSON Object. I just want to point out
that it cannot be the final solution from the point of a USER as it implies
horrible performance drawbacks.

(2) A default parser bundled with pgjdbc simply makes no sense as (a) in a
Java EE environment a JSONP API provider IS found on the classpath already,
and (b) it is not unlikely that the same will be the case for Java SE 9 due
to the implied importance of JSON these days, and (c) an application
programmer dealing with JSON has to have a JSON parser anyways, even if he
does not use JDBC at all.

-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Samstag, 27. Juni 2015 23:30
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 27/06/15 20:07, Markus KARG wrote:
> Álvaro,
>
> I also think we should look at the topic with the eyes of the user, but I
> have to disagree in two point exactly due to that attitude:
     Hi Markus. Good that we agree on that :)
>
> (1) JSON is definitively not just another datatype. Just like XML, it
> provides the ability to return a complete object graph as the content of
one
> column of one row. Hence, it contains the same amount of data that in
> pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
> words, from the complexity of the information and from the aspect of end
> user performance, it "feels" like a ResulSet that contains ResultSets that
> contains ResultSets... Handling it just like any other datatype is what
> pgjdbc does currently, and it provides very poor performance due to that,
as
> it simply ships the complete graph - independent of the fact whether the
end
> user will process is completely or just pick a single node of it. Asking
to
> handle JSON as just any other data type is like asking to give up
ResultSet
> in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
> of an end user, the answer must be, to NOT handle JSON like any other
> datatype, and NOT return a JsonObject, but instead provide a streaming
API,
> just like SQLXML does. This allows to only transfer and process that few
> nodes the the end user actually likes to have, just like an end user
> typically will never process a complete ResultSet but process it
iteratively
> and most typically filtered.
     When I said "as just another datatype", I mean "as simple as". So
from a user perspective I would expect to have a get...JSON() method
with return value an object similar to (or IMHO exactly) JsonObject. In
other words, something directly usable.

     As Dave Cramer correctly points out in response earlier in this
thread, there is no way in current PostgreSQL support to send a partial
chunk of the JSON, so --unfortunately-- streaming API is not possible.
Not being it possible, why not provide methods to return JsonObject, the
Java EE standard and defined by a JSR?

>
> (2) The user MUST be always asked to provide a JSON parser, as it is HIM
who
> wants to use JSON, and so it is HIS choice to select one among those
> fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc
picks
> and / or provides one. Most typically this will end up in the same design
> choice than most XML applications ended up with: Relying on Java SE / EE
> providing a default product, like it is the case with JAXB (Xerces). The
> only thing pgjdbc can really do NOW is to support the JSON SPI *if* there
is
> a processor on the classpath, and once the postgresql server produces an
> improved streaming procotol for JSON / XML we can couple the client's JSON
> event handler with the server's protocol events.

     Why not instead provide a default parser and let the user choose
another one if so she wants? It makes life easier for 90% of users who
don't care or even know about which parser to choose. Providing a
sensible default (or just a working default) is better than providing none.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
> Regards
> Markus
>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Samstag, 27. Juni 2015 18:56
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
> On 26/06/15 19:29, Steven Schlansker wrote:
>> On Jun 26, 2015, at 10:23 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>
>>> On 26 June 2015 at 13:01, Steven Schlansker <stevenschlansker@gmail.com>
> wrote:
>>> On Jun 26, 2015, at 7:57 AM, Dave Cramer <davecramer@gmail.com> wrote:
>>>
>>>> I'm looking for comments on how to implement a SQLJSON type in JDBC.
>>>>
>>>> As there is no getSQLJSON in the resultset interface this could only be
> used in getObject.
>>>> Notionally it would model itself after SQLXML.
>>>> https://docs.oracle.com/javase/7/docs/api/index.html?java/sql/SQLXML
>>>> .html
>>> I used JSON extensively in one of my projects, but have never used
> SQLXML.  I'm having trouble understanding why the SQLXML interface adds
any
> value to passing rs.getBinaryStream to your favorite JSON parser.
> Especially since you would have to use getObject, I am not seeing how:
>>> rs.getObject("field", SQLJSON.class).mapToType(MyType.class)
>>>
>>> is simpler than:
>>> jacksonObjectMapper.readValue(rs.getBinaryStream("field"),
>>> MyType.class)
>>>
>>> which already works today as far as I understand.  Doubly so since
nobody
> will agree on which JSON parsing library to use.
>>> I'm sure I'm missing something?
>>>
>>>
>>> I don't think you are; as you rightly pointed out now we would have
>>> to add a json parser to the driver, which I'm reluctant to do
>>>
>> If this feature is developed, I think the JSON parser should be pluggable
> and optional if possible.  Then users that do not want it do not need to
> drag in a large dependency.
>> That said, without a more convincing use case or a compelling API that we
> could easily add, I don't see this interface being "worth its weight" as
an
> addition.
>>
>>
>       Hi List.
>
>       I always try to think more from the user perspective than from the
> developer one. This is also going to be the case.
>
>       Having JSON support in the PostgreSQL JDBC driver is a *must*.
> jsonb was 9.4's next-big-thing-since-sliced-bread jet there's no support
in
> one of the most used PostgreSQL drivers. No blame here (at all), just
trying
> to support my point here.
>
>       I don't see the advantage of using SQLJSON, although I wouldn't
argue
> against. What I clearly believe is that at the end of the day you should
be
> able to easily return a javax.json.JsonObject from a ResultSet. Being a
> JavaEE standard and a JSR, I believe it's the best (and obvious) choice.
>
>       Regarding pluggability, JSR353's SPI mechanism is good, but asking
the
> user to provide a further dependency "just for reading JSON" seems again
to
> me not good from the user perspective. I'd ideally expect json to be
> supported as-is, as with any other datatype. Having the SPI we could
choose
> whatever implementation we want. Is that enlarging the driver's size? So
> what? Users want easy-of-use, not driver size. And there are many
mechanisms
> to reduce size for unused classes.
>
>       If any, my 2 cents are: let's add JSON, let's take JSR353 as an API
for
> it and let's make it as easy as possible for final users to use it.
>
>       Regards,
>
>       Álvaro
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Sehrope Sarkuni
Date:
Thinking this through a bit, I see a couple things we'd need to figure out:

1. What parser would we use?
2. What object type(s) would the parser return?
3. What would the end-user API look like?
4. What's the end user benefit of this?

#1 has lots of possible answers. Everything from pluggable APIs, classpath scanning, the JSON APIs mentioned so far in this thread.

#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...

#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing. 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 00:10, Markus KARG wrote:
> (1) I have no problem with providing a JSON Object. I just want to point out
> that it cannot be the final solution from the point of a USER as it implies
> horrible performance drawbacks.
     Those performance problems are inevitable as of today. Unless you
change PostgreSQL's protocol (and you will have me by your side on that
one) there's nothing we can do. So with the current status quo, there's
nothing else to do. And so the best we can do (today) is provide that
JsonObject.

>
> (2) A default parser bundled with pgjdbc simply makes no sense as (a) in a
> Java EE environment a JSONP API provider IS found on the classpath already,
     We may provide classloading mechanism to opt-out or different
packaging alternatives (a JDBC driver for EE and one for standalone).
There may be more alternatives. But not providing one is making our
user's life more complicated.

> and (b) it is not unlikely that the same will be the case for Java SE 9 due
> to the implied importance of JSON these days,
     Java9 is far and away. And it will change the way dynamic code
loading happens. I'm all in to refactor the driver for Java9 and take
advantage of modules, and that surely helps here, but we need to look
now at the present.

>   and (c) an application
> programmer dealing with JSON has to have a JSON parser anyways, even if he
> does not use JDBC at all.
     That I don't agree with. You may store data in json format in
PostgreSQL and then convert to your domain objects which may or may not
use JSON. So we cannot assume the user already has a JSON parser.

     The best we can do is that the JDBC driver works by default, rather
than throwing an exception because there is no JSON parser loaded. We
have to make it easy for the users.

     Best,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Samstag, 27. Juni 2015 23:30
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
> On 27/06/15 20:07, Markus KARG wrote:
>> Álvaro,
>>
>> I also think we should look at the topic with the eyes of the user, but I
>> have to disagree in two point exactly due to that attitude:
>       Hi Markus. Good that we agree on that :)
>> (1) JSON is definitively not just another datatype. Just like XML, it
>> provides the ability to return a complete object graph as the content of
> one
>> column of one row. Hence, it contains the same amount of data that in
>> pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
>> words, from the complexity of the information and from the aspect of end
>> user performance, it "feels" like a ResulSet that contains ResultSets that
>> contains ResultSets... Handling it just like any other datatype is what
>> pgjdbc does currently, and it provides very poor performance due to that,
> as
>> it simply ships the complete graph - independent of the fact whether the
> end
>> user will process is completely or just pick a single node of it. Asking
> to
>> handle JSON as just any other data type is like asking to give up
> ResultSet
>> in favor of returning an ArrayList<ArrayList<?>>! So looking with the eyes
>> of an end user, the answer must be, to NOT handle JSON like any other
>> datatype, and NOT return a JsonObject, but instead provide a streaming
> API,
>> just like SQLXML does. This allows to only transfer and process that few
>> nodes the the end user actually likes to have, just like an end user
>> typically will never process a complete ResultSet but process it
> iteratively
>> and most typically filtered.
>       When I said "as just another datatype", I mean "as simple as". So
> from a user perspective I would expect to have a get...JSON() method
> with return value an object similar to (or IMHO exactly) JsonObject. In
> other words, something directly usable.
>
>       As Dave Cramer correctly points out in response earlier in this
> thread, there is no way in current PostgreSQL support to send a partial
> chunk of the JSON, so --unfortunately-- streaming API is not possible.
> Not being it possible, why not provide methods to return JsonObject, the
> Java EE standard and defined by a JSR?
>
>> (2) The user MUST be always asked to provide a JSON parser, as it is HIM
> who
>> wants to use JSON, and so it is HIS choice to select one among those
>> fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc
> picks
>> and / or provides one. Most typically this will end up in the same design
>> choice than most XML applications ended up with: Relying on Java SE / EE
>> providing a default product, like it is the case with JAXB (Xerces). The
>> only thing pgjdbc can really do NOW is to support the JSON SPI *if* there
> is
>> a processor on the classpath, and once the postgresql server produces an
>> improved streaming procotol for JSON / XML we can couple the client's JSON
>> event handler with the server's protocol events.
>       Why not instead provide a default parser and let the user choose
> another one if so she wants? It makes life easier for 90% of users who
> don't care or even know about which parser to choose. Providing a
> sensible default (or just a working default) is better than providing none.
>
>       Regards,
>
>       Álvaro
>
>



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 28/06/15 00:11, Sehrope Sarkuni wrote:
> Thinking this through a bit, I see a couple things we'd need to figure
> out:
>
> 1. What parser would we use?
> 2. What object type(s) would the parser return?
> 3. What would the end-user API look like?
> 4. What's the end user benefit of this?
>
> #1 has lots of possible answers. Everything from pluggable APIs,
> classpath scanning, the JSON APIs mentioned so far in this thread.

     Hi Sehrope!

     To me, this is the least important question. If based on JSR353's
SPI, it's trivial to swap the default, included one, for another one.
Just picking a sensible default (Jackson, for instance) is probably good
enough.

>
> #2 is driven a lot by #1 as depending on the parser implementation
> there may be different object types returned. JSON is a bit tricky as
> "valid JSON" can mean null, a scalar, an object, or an array. Most
> people thing of it as just an object but "foobar" is valid JSON as
> well. This leads us to #3...

     The object type to return has to be IMHO JsonObject:
http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

>
> #3 doesn't have a straight answer as there is no getJSON(...) methods
> in the JDBC spec. It'd probably have to be returned via getObject(...).
>
> An alternative is to provide PGResultSet and PGPreparedStatement
> classes similar to PGConnection that provides PG extensions. They
> could have the get/set methods (ex: getJsonScalar(...) or
> setJsonObject(Map<String,Object> ...)) to retrieve JSON values as
> specific object types (i.e. scalar, object, array). It'd be a bit more
> type safe as presumably most people using json/jsonb types know the
> top level type of what they're storing.

     Probably adding methods to PG classes would be better than
getObject and force explicit casts. Regarding the methods, if they
simply return JsonObject, you already have a full API there to parse and
extract and process. So anything that returns a JsonObject from a column
(identifier or #) would be enough for me.

>
> For #4 I see two possible wins. First off on the usability side,
> there's some convenience to natively interfacing with json/jsonb
> types. It'll only have value though if those types are the same ones
> that users are using in the rest of their code. If they're just using
> them as Map<String,Object> everywhere then it'd still be a pain for a
> user to convert to our "native" PG JSON types to use via JDBC. Having
> a dedicated API that allows for interaction using native Java types
> would make this more convenient.
>
> The other win I can see for #4 is on performance. Right now JSON is
> converted to a String. That means everybody using it has to convert it
> twice. First raw bytes to String, then String to object. A dedicated
> API could cut one of those out of the way. Given how the wire protocol
> is implemented in the driver, it wouldn't be a direct reading from the
> input stream (it'll be buffered in a byte array), but at least it
> won't be copied twice.

     As far as I know, most users are using JsonObject, so returning
that is a perfect match for pgjdbc. I don't expect however big
performance wins as JSON is sent as a String over the wire...

     Regards,

     Álvaro

--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
Sehrope Sarkuni
Date:
On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
    Hi Sehrope!

Hi Álvaro! :D
 
    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...

    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
 
Not always though. All these are valid JSON too: 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
 num | string |   arr   |      obj      
-----+--------+---------+---------------
 1   | "test" | [1,2,3] | {"foo":"bar"}
(1 row)

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).
 

#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.

    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific. 
 
For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.

    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 00:55, Sehrope Sarkuni wrote:
On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
    Hi Sehrope!

Hi Álvaro! :D
 
    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)

    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).

    JSR353 is targeted for JavaSE 6 :)



#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...

    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
 
Not always though. All these are valid JSON too: 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
 num | string |   arr   |      obj      
-----+--------+---------+---------------
 1   | "test" | [1,2,3] | {"foo":"bar"}
(1 row)

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).

    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)
 

#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.

    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.

    Yepp

 
For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.

    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.

    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro

-- 
Álvaro Hernández Tortosa


-----------
8Kdata

Re: SQLJSON

From
Christopher BROWN
Date:
Hello,

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.
  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).
  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).
  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future. 
--
Christopher


On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 00:55, Sehrope Sarkuni wrote:
On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
    Hi Sehrope!

Hi Álvaro! :D
 
    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)

    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).

    JSR353 is targeted for JavaSE 6 :)



#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...

    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
 
Not always though. All these are valid JSON too: 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
 num | string |   arr   |      obj      
-----+--------+---------+---------------
 1   | "test" | [1,2,3] | {"foo":"bar"}
(1 row)

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).

    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)
 

#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.

    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.

    Yepp

 
For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.

    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.

    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro

-- 
Álvaro Hernández Tortosa


-----------
8Kdata


Re: SQLJSON

From
"Markus KARG"
Date:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for JSON
and / or XML.


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 00:19
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON



On 28/06/15 00:10, Markus KARG wrote:
> (1) I have no problem with providing a JSON Object. I just want to point
out
> that it cannot be the final solution from the point of a USER as it
implies
> horrible performance drawbacks.
     Those performance problems are inevitable as of today. Unless you
change PostgreSQL's protocol (and you will have me by your side on that
one) there's nothing we can do. So with the current status quo, there's
nothing else to do. And so the best we can do (today) is provide that
JsonObject.

>
> (2) A default parser bundled with pgjdbc simply makes no sense as (a) in a
> Java EE environment a JSONP API provider IS found on the classpath
already,
     We may provide classloading mechanism to opt-out or different
packaging alternatives (a JDBC driver for EE and one for standalone).
There may be more alternatives. But not providing one is making our
user's life more complicated.

> and (b) it is not unlikely that the same will be the case for Java SE 9
due
> to the implied importance of JSON these days,
     Java9 is far and away. And it will change the way dynamic code
loading happens. I'm all in to refactor the driver for Java9 and take
advantage of modules, and that surely helps here, but we need to look
now at the present.

>   and (c) an application
> programmer dealing with JSON has to have a JSON parser anyways, even if he
> does not use JDBC at all.
     That I don't agree with. You may store data in json format in
PostgreSQL and then convert to your domain objects which may or may not
use JSON. So we cannot assume the user already has a JSON parser.

     The best we can do is that the JDBC driver works by default, rather
than throwing an exception because there is no JSON parser loaded. We
have to make it easy for the users.

     Best,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Samstag, 27. Juni 2015 23:30
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
> On 27/06/15 20:07, Markus KARG wrote:
>> Álvaro,
>>
>> I also think we should look at the topic with the eyes of the user, but I
>> have to disagree in two point exactly due to that attitude:
>       Hi Markus. Good that we agree on that :)
>> (1) JSON is definitively not just another datatype. Just like XML, it
>> provides the ability to return a complete object graph as the content of
> one
>> column of one row. Hence, it contains the same amount of data that in
>> pre-XML and pre-JSON times, the ResultSet did handle alone. Or in other
>> words, from the complexity of the information and from the aspect of end
>> user performance, it "feels" like a ResulSet that contains ResultSets
that
>> contains ResultSets... Handling it just like any other datatype is what
>> pgjdbc does currently, and it provides very poor performance due to that,
> as
>> it simply ships the complete graph - independent of the fact whether the
> end
>> user will process is completely or just pick a single node of it. Asking
> to
>> handle JSON as just any other data type is like asking to give up
> ResultSet
>> in favor of returning an ArrayList<ArrayList<?>>! So looking with the
eyes
>> of an end user, the answer must be, to NOT handle JSON like any other
>> datatype, and NOT return a JsonObject, but instead provide a streaming
> API,
>> just like SQLXML does. This allows to only transfer and process that few
>> nodes the the end user actually likes to have, just like an end user
>> typically will never process a complete ResultSet but process it
> iteratively
>> and most typically filtered.
>       When I said "as just another datatype", I mean "as simple as". So
> from a user perspective I would expect to have a get...JSON() method
> with return value an object similar to (or IMHO exactly) JsonObject. In
> other words, something directly usable.
>
>       As Dave Cramer correctly points out in response earlier in this
> thread, there is no way in current PostgreSQL support to send a partial
> chunk of the JSON, so --unfortunately-- streaming API is not possible.
> Not being it possible, why not provide methods to return JsonObject, the
> Java EE standard and defined by a JSR?
>
>> (2) The user MUST be always asked to provide a JSON parser, as it is HIM
> who
>> wants to use JSON, and so it is HIS choice to select one among those
>> fulfilling the JSONP API standard. Hence it makes no sense that pgjdbc
> picks
>> and / or provides one. Most typically this will end up in the same design
>> choice than most XML applications ended up with: Relying on Java SE / EE
>> providing a default product, like it is the case with JAXB (Xerces). The
>> only thing pgjdbc can really do NOW is to support the JSON SPI *if* there
> is
>> a processor on the classpath, and once the postgresql server produces an
>> improved streaming procotol for JSON / XML we can couple the client's
JSON
>> event handler with the server's protocol events.
>       Why not instead provide a default parser and let the user choose
> another one if so she wants? It makes life easier for 90% of users who
> don't care or even know about which parser to choose. Providing a
> sensible default (or just a working default) is better than providing
none.
>
>       Regards,
>
>       Álvaro
>
>



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 09:34, Christopher BROWN wrote:
Hello,

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.
  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).
    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.
  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).
    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.

  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.
    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata




--
Christopher


On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 00:55, Sehrope Sarkuni wrote:
On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
    Hi Sehrope!

Hi Álvaro! :D
 
    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)

    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).

    JSR353 is targeted for JavaSE 6 :)



#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...

    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
 
Not always though. All these are valid JSON too: 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
 num | string |   arr   |      obj      
-----+--------+---------+---------------
 1   | "test" | [1,2,3] | {"foo":"bar"}
(1 row)

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).

    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)
 

#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.

    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.

    Yepp

 
For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.

    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.

    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro

-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 28/06/15 11:17, Markus KARG wrote:
> I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
> and then parsing it on behalf of the application is pretty simple and
> straightforward. My vote would be to not do anything until JDBC 4.3 of JDBC
> 5.0 provides a standard API for dealing with JSON inside of the driver or at
> least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for JSON
> and / or XML.
     Don't do anything?

     And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

     Really?

     User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Sonntag, 28. Juni 2015 00:19
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
> On 28/06/15 00:10, Markus KARG wrote:
>> (1) I have no problem with providing a JSON Object. I just want to point
> out
>> that it cannot be the final solution from the point of a USER as it
> implies
>> horrible performance drawbacks.
>       Those performance problems are inevitable as of today. Unless you
> change PostgreSQL's protocol (and you will have me by your side on that
> one) there's nothing we can do. So with the current status quo, there's
> nothing else to do. And so the best we can do (today) is provide that
> JsonObject.
>
>> (2) A default parser bundled with pgjdbc simply makes no sense as (a) in a
>> Java EE environment a JSONP API provider IS found on the classpath
> already,
>       We may provide classloading mechanism to opt-out or different
> packaging alternatives (a JDBC driver for EE and one for standalone).
> There may be more alternatives. But not providing one is making our
> user's life more complicated.
>
>> and (b) it is not unlikely that the same will be the case for Java SE 9
> due
>> to the implied importance of JSON these days,
>       Java9 is far and away. And it will change the way dynamic code
> loading happens. I'm all in to refactor the driver for Java9 and take
> advantage of modules, and that surely helps here, but we need to look
> now at the present.
>
>>    and (c) an application
>> programmer dealing with JSON has to have a JSON parser anyways, even if he
>> does not use JDBC at all.
>       That I don't agree with. You may store data in json format in
> PostgreSQL and then convert to your domain objects which may or may not
> use JSON. So we cannot assume the user already has a JSON parser.
>
>       The best we can do is that the JDBC driver works by default, rather
> than throwing an exception because there is no JSON parser loaded. We
> have to make it easy for the users.
>
>       Best,
>
>       Álvaro
>
>



Re: SQLJSON

From
"Markus KARG"
Date:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.

  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.


  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)


 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)

 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp


 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

Re: SQLJSON

From
Christopher BROWN
Date:
Regarding JSONP support with PostgreSQL's JDBC driver, instead of lots of inference about what's best, maybe we could just use connection properties (to enable getObject() usage for retrieving either a JSON Model, or Stream, and to specify the JSR-353 implementation to load) ?

Re: SQLJSON

From
"Markus KARG"
Date:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.

-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
> I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
> and then parsing it on behalf of the application is pretty simple and
> straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
> 5.0 provides a standard API for dealing with JSON inside of the driver or
at
> least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
> and / or XML.
     Don't do anything?

     And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

     Really?

     User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Sonntag, 28. Juni 2015 00:19
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
> On 28/06/15 00:10, Markus KARG wrote:
>> (1) I have no problem with providing a JSON Object. I just want to point
> out
>> that it cannot be the final solution from the point of a USER as it
> implies
>> horrible performance drawbacks.
>       Those performance problems are inevitable as of today. Unless you
> change PostgreSQL's protocol (and you will have me by your side on that
> one) there's nothing we can do. So with the current status quo, there's
> nothing else to do. And so the best we can do (today) is provide that
> JsonObject.
>
>> (2) A default parser bundled with pgjdbc simply makes no sense as (a) in
a
>> Java EE environment a JSONP API provider IS found on the classpath
> already,
>       We may provide classloading mechanism to opt-out or different
> packaging alternatives (a JDBC driver for EE and one for standalone).
> There may be more alternatives. But not providing one is making our
> user's life more complicated.
>
>> and (b) it is not unlikely that the same will be the case for Java SE 9
> due
>> to the implied importance of JSON these days,
>       Java9 is far and away. And it will change the way dynamic code
> loading happens. I'm all in to refactor the driver for Java9 and take
> advantage of modules, and that surely helps here, but we need to look
> now at the present.
>
>>    and (c) an application
>> programmer dealing with JSON has to have a JSON parser anyways, even if
he
>> does not use JDBC at all.
>       That I don't agree with. You may store data in json format in
> PostgreSQL and then convert to your domain objects which may or may not
> use JSON. So we cannot assume the user already has a JSON parser.
>
>       The best we can do is that the JDBC driver works by default, rather
> than throwing an exception because there is no JSON parser loaded. We
> have to make it easy for the users.
>
>       Best,
>
>       Álvaro
>
>



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
"Markus KARG"
Date:

I'd call this "configuration hell". ;-)

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Christopher BROWN
Sent: Sonntag, 28. Juni 2015 09:49
To: List
Subject: Re: [JDBC] SQLJSON

 

Regarding JSONP support with PostgreSQL's JDBC driver, instead of lots of inference about what's best, maybe we could just use connection properties (to enable getObject() usage for retrieving either a JSON Model, or Stream, and to specify the JSR-353 implementation to load) ?

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.

  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.


  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)


 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)

 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp


 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 28/06/15 11:51, Markus KARG wrote:
> It is not *us* who let the JSON users down, it is the PostgreSQL protocol
> guys who did not add any useful support for JSON. A driver is not a
> compensation for missing product features, it is just a thin wrapper around
> the base product's features.
     To have proper JSON support at the protocol level (something which
I'd love to have) only translates to more performance, no more
functionality. So is a nice-to-have, not a must-to-have (as is
supporting PostgreSQL's json data types).
>
> I mean, what happens if the application shall work with a different product?
> If you rely on non-JDBC-features, you're screwed. So a profession
> application using JSON should ALWAYS come with JSR 253 anyways.
     We have had to extend JDBC in several ways in the past. We should
do it again, now, in the best possible manner (getObject, PGResultSet,
whatever). And then, if JDBC adds support in the future, retrofit into
it. But not wait until then, because we don't even know if that would
even happen.

     Cheers,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
> -----Original Message-----
> From: pgsql-jdbc-owner@postgresql.org
> [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
> Tortosa
> Sent: Sonntag, 28. Juni 2015 11:44
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
> On 28/06/15 11:17, Markus KARG wrote:
>> I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
>> and then parsing it on behalf of the application is pretty simple and
>> straightforward. My vote would be to not do anything until JDBC 4.3 of
> JDBC
>> 5.0 provides a standard API for dealing with JSON inside of the driver or
> at
>> least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
> JSON
>> and / or XML.
>       Don't do anything?
>
>       And let Java PostgreSQL users down, without a (driver, supported)
> means of getting JSON out of their database? So we make the "marketing"
> that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
> JSON with Java?
>
>       Really?
>
>       User's don't care about extreme performance. Users care about easy
> of use and decent set of features. Adding JSON support, even thought
> it's not the most performant one is something we should be doing as
> quickly as possible.
>
>       Regards,
>
>       Álvaro
>
>



Re: SQLJSON

From
Dave Cramer
Date:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
"Markus KARG"
Date:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

Re: SQLJSON

From
"Markus KARG"
Date:

Wouldn't it be more clever to ask the pgjdbc *users* what *they* expect to get instead of solely relying upon us three? As the discussion turned out, we have fundamentally diverse opinions, and I doubt that we will do it any right if we just do "something" simply for the benefit of writing "JSON support" on our driver.

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Sonntag, 28. Juni 2015 15:56
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

 

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


On 28/06/15 11:51, Markus KARG wrote:

It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.

    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).


I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.

    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,



    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:

I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of

JDBC

5.0 provides a standard API for dealing with JSON inside of the driver or

at

least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for

JSON

and / or XML.

      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro




--

Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 

Re: SQLJSON

From
Dave Cramer
Date:
Hi Markus,

While I think that would be a wonderful idea, my experience in the last 15 years of working with this driver is that nobody will respond to your question. That being said feel free to take a poll, or otherwise solicit responses. 

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 11:13, Markus KARG <markus@headcrashing.eu> wrote:

Wouldn't it be more clever to ask the pgjdbc *users* what *they* expect to get instead of solely relying upon us three? As the discussion turned out, we have fundamentally diverse opinions, and I doubt that we will do it any right if we just do "something" simply for the benefit of writing "JSON support" on our driver.

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Sonntag, 28. Juni 2015 15:56
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

 

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


On 28/06/15 11:51, Markus KARG wrote:

It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.

    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).


I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.

    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,



    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:

I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of

JDBC

5.0 provides a standard API for dealing with JSON inside of the driver or

at

least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for

JSON

and / or XML.

      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro




--

Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata


 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc


Re: SQLJSON

From
Christopher BROWN
Date:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

Le 28 juin 2015 22:11, "Álvaro Hernández Tortosa" <aht@8kdata.com> a écrit :

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc


Re: SQLJSON

From
Dave Cramer
Date:



On 28 June 2015 at 16:32, Christopher BROWN <brown@reflexe.fr> wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.

The API, or the implementation ???  

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .


so how do we make it "Just Work" ?


Dave Cramer

dave.cramer(at)credativ(dot)ca
Le 28 juin 2015 22:11, "Álvaro Hernández Tortosa" <aht@8kdata.com> a écrit :

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 22:32, Christopher BROWN wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.


    Hi Christopher. May you elaborate on which conflicts may it create? The API is quite stable, so I don't see a chance for conflicts. The RI may be a more likely source of conflicts, yet I see them quite unlikely. And if so, if it would be so important to avoid them, different packages may be provided.

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

    I'm no OSGI expert, so I may fail to understand this properly, but how problematic this is? Is hot reloading the JDBC driver a big issue?

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

    So if there are problems even bundling the API, what solution do you suggest to provide end users with a facility to read their new, fancy and much hyped jsonb columns?

    If there is none, I'd definitely study what the percentage of people would have problems with the proposed approach (API + RI?).

    Regards,

    Álvaro
-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Le 28 juin 2015 22:11, "Álvaro Hernández Tortosa" <aht@8kdata.com> a écrit :

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc




			
		

Re: SQLJSON

From
Sehrope Sarkuni
Date:
On Sun, Jun 28, 2015 at 4:35 PM, Dave Cramer <pg@fastcrypt.com> wrote:
On 28 June 2015 at 16:32, Christopher BROWN <brown@reflexe.fr> wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.

The API, or the implementation ???  

Both. Embedding dependencies is mildly convenient for someone getting started with something but royally inconvenient for someone with a complicated environment. Dependency management systems like Maven handle this pretty well but even they can't deal with situations where classes are embedded in a dependency (ex: javax.json.* embedded within the PG JDBC driver).

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

so how do we make it "Just Work" ?
 
I don't think there's a way to do this without breaking backward compatibility.

You definitely can't rely on specific dependencies on the classpath that legacy users will not have. Bundling the dependencies doesn't work either as it'd clobber existing ones on the classpath. Making things dynamic/pluggable may be option for internal implementations but it breaks anything with the required dependencies in the method signature. That means we could have code that dynamically picks a JSON parser and uses it internally, but we can't have a getJsonValue() method on a public PGResultSet interface as the class wouldn't even load properly on an older JVM[1].

In a lot of ways this is similar to the other thread we had about dropping support for older JVMs. To natively support new features (like a native getJsonValue()) we'd need to specify a min JDK version.

At this point I think we have to bite bullet and either drop support for older versions (not likely) or offer multiple versions of the driver. The latter could make modern assumptions about the classpath/environment to support new features natively. That could include JsonValue and the new Java 8 date/time types.

[1]: PGResultSet doesn't exist yet so that may be a bad example as presumably any user of it would be a "new" user. PGStatement does exist though so arguably we'd have the reverse problem there for a setJsonValue(...) method.

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Re: SQLJSON

From
Christopher BROWN
Date:
As we've seen from a recent discussion on this list, there's apparently a lot of users still downloading old versions of the driver.  If you embed a version of the API, then (unless a user derives their own version of the driver), you prevent the user from independently changing the JSONP version.  The RI is even more subject to this, as it may require updates to fix bugs.

Anyone compiling client code requiring use of the JSONP API will need to have that API in their compiler classpath.  Many of the "make it just work" users are likely to compile their JDBC code against the JDBC classes and interfaces in the JDK classpath without requiring to add the PostgreSQL JDBC driver to the compiler classpath.  They may or may not already have a version of the JSONP API in their classpath (probably will if compiling for a Java EE application server, not so likely otherwise).  They will of course need the PostgreSQL JDBC driver in their classpath for compiling if they use any PGxxx classes.

The general classloading strategy mandated is that parent classloaders should be used to load classes before child classloaders.  Many application servers however will lookup classes from embedded libraries (WEB-INF/lib) first, but this varies from one server to another, and often based on configuration options.  OSGi (and possibly Java9 modules) allow classloading from classloader graphs.  In any case, suppose that your application container defines a more recent version of the API than the driver: you'll need to hack the driver to be able to use the newer APIs.  Suppose that the driver embeds a more recent version of the API: you'll get classloading errors because you refer to methods or fields that don't exist in the application server's version.  And of course, if some other helpful library also embeds the JSONP API (or you have an alternative to the RI in your classpath, expecting a different version of the API), say a web services library or a persistence in WEB-INF/lib, you'll have other types of errors.

One of my colleagues is already using JSONB with PostgreSQL, by reading and writing stringified JSON using Jackson.  I don't know how a meaningful (accurate) study could be conducted for more insight into what prospective users would select, especially as that would in turn depend upon what is decided here.  Self-fulfilling prophecies, etc.

It would make sense to make few assumptions about what getObject() should return, because in some cases you'd want an Object Model view, and in other cases access to a parser or generator (because of improved performance).  It would make sense for me to be able to cast to a PGxxx object, in turn providing access to either a fully-constructed JSON object tree, or as a push/pull parser.  In any case, the PostgreSQL JSONB support isn't SQL-neutral, as it defines custom operators and lots of specific stuff anyway (the PGxxx object via getObject() to make it accessible via JDBC pools that wrap statements and resultsets).  You currently can't write database-neutral JSON/SQL, so until JSON is standardised at SQL level, and at JDBC level, I can't see anything wrong with a specific driver dependency.

Performance is better through streaming APIs because you create less objects, using less memory, with less GC pressure.  You can also skip through lots of data thereby avoiding lots of heap allocations.

The solution that would make most sense, would be to provide a version that embeds the API + RI, and a version that doesn't (the latter version would be compiled in the same way, it would just require that the API be made available by the host application).

--
Christopher


On 28 June 2015 at 22:40, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 22:32, Christopher BROWN wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.


    Hi Christopher. May you elaborate on which conflicts may it create? The API is quite stable, so I don't see a chance for conflicts. The RI may be a more likely source of conflicts, yet I see them quite unlikely. And if so, if it would be so important to avoid them, different packages may be provided.

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

    I'm no OSGI expert, so I may fail to understand this properly, but how problematic this is? Is hot reloading the JDBC driver a big issue?

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

    So if there are problems even bundling the API, what solution do you suggest to provide end users with a facility to read their new, fancy and much hyped jsonb columns?

    If there is none, I'd definitely study what the percentage of people would have problems with the proposed approach (API + RI?).


    Regards,

    Álvaro
-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Le 28 juin 2015 22:11, "Álvaro Hernández Tortosa" <aht@8kdata.com> a écrit :

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc




Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 23:11, Christopher BROWN wrote:
As we've seen from a recent discussion on this list, there's apparently a lot of users still downloading old versions of the driver.  If you embed a version of the API, then (unless a user derives their own version of the driver), you prevent the user from independently changing the JSONP version.  The RI is even more subject to this, as it may require updates to fix bugs.

Anyone compiling client code requiring use of the JSONP API will need to have that API in their compiler classpath.  Many of the "make it just work" users are likely to compile their JDBC code against the JDBC classes and interfaces in the JDK classpath without requiring to add the PostgreSQL JDBC driver to the compiler classpath.  They may or may not already have a version of the JSONP API in their classpath (probably will if compiling for a Java EE application server, not so likely otherwise).  They will of course need the PostgreSQL JDBC driver in their classpath for compiling if they use any PGxxx classes.

The general classloading strategy mandated is that parent classloaders should be used to load classes before child classloaders.  Many application servers however will lookup classes from embedded libraries (WEB-INF/lib) first, but this varies from one server to another, and often based on configuration options.  OSGi (and possibly Java9 modules) allow classloading from classloader graphs.  In any case, suppose that your application container defines a more recent version of the API than the driver: you'll need to hack the driver to be able to use the newer APIs.  Suppose that the driver embeds a more recent version of the API: you'll get classloading errors because you refer to methods or fields that don't exist in the application server's version.  And of course, if some other helpful library also embeds the JSONP API (or you have an alternative to the RI in your classpath, expecting a different version of the API), say a web services library or a persistence in WEB-INF/lib, you'll have other types of errors.

One of my colleagues is already using JSONB with PostgreSQL, by reading and writing stringified JSON using Jackson.  I don't know how a meaningful (accurate) study could be conducted for more insight into what prospective users would select, especially as that would in turn depend upon what is decided here.  Self-fulfilling prophecies, etc.

It would make sense to make few assumptions about what getObject() should return, because in some cases you'd want an Object Model view, and in other cases access to a parser or generator (because of improved performance).  It would make sense for me to be able to cast to a PGxxx object, in turn providing access to either a fully-constructed JSON object tree, or as a push/pull parser.  In any case, the PostgreSQL JSONB support isn't SQL-neutral, as it defines custom operators and lots of specific stuff anyway (the PGxxx object via getObject() to make it accessible via JDBC pools that wrap statements and resultsets).  You currently can't write database-neutral JSON/SQL, so until JSON is standardised at SQL level, and at JDBC level, I can't see anything wrong with a specific driver dependency.

Performance is better through streaming APIs because you create less objects, using less memory, with less GC pressure.  You can also skip through lots of data thereby avoiding lots of heap allocations.

The solution that would make most sense, would be to provide a version that embeds the API + RI, and a version that doesn't (the latter version would be compiled in the same way, it would just require that the API be made available by the host application).

    Hi Christopher.

    I don't have any objection to building two different packages or driver versions, if that makes sense.

    However, I don't see the point you make about the API. Although correct, it's only so from a theoretical perspective. The API is meant to be stable, and it will surely be for quite some years. And if it changes, it will be a substantial change that would require linking with the correct library and surely a code change. A bigger problem than dealing with classloaders. But until then (if that ever happens), the API will not change, and thus including it in the JDBC driver will never conflict with that being included in any transitive dependency, as it would be the same version (and if it changes, a new driver version would be required anyway).

    The RI admittedly is more subject to change (bug fixes, performance improvements), but being a RI:

- It's not expected to change frequently
- Few would be using it anyway

    So I don't expect those collisions to be frequent either.

    All in all, while theoretically true, we need to think practically: what is the likelihood of this happening with the API? I'd say zero. With the RI? Low enough. Thus I'd follow the idea of two separate packages, but one with the API the other with API+RI, having the latter become the default version of the driver and calling the former something like the "non-json-parser" version.

     Regarding compile-time dependencies, having the API on the driver, also non-JavaEE7 users would be happy just by including the driver as a compile-time dependency. Not a huge deal. And JavaEE7 users would be serviced without this need.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata




--
Christopher


On 28 June 2015 at 22:40, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 22:32, Christopher BROWN wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.


    Hi Christopher. May you elaborate on which conflicts may it create? The API is quite stable, so I don't see a chance for conflicts. The RI may be a more likely source of conflicts, yet I see them quite unlikely. And if so, if it would be so important to avoid them, different packages may be provided.

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

    I'm no OSGI expert, so I may fail to understand this properly, but how problematic this is? Is hot reloading the JDBC driver a big issue?

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

    So if there are problems even bundling the API, what solution do you suggest to provide end users with a facility to read their new, fancy and much hyped jsonb columns?

    If there is none, I'd definitely study what the percentage of people would have problems with the proposed approach (API + RI?).


    Regards,

    Álvaro
-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Le 28 juin 2015 22:11, "Álvaro Hernández Tortosa" <aht@8kdata.com> a écrit :

On 28/06/15 15:56, Dave Cramer wrote:
So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.

    Yes, at least to have a JsonValue would be a really nice addition.

    To plug your parser, JSR 353 follows Java's standard SPI and is as simple as write the fqcn of the driver implementation to META-INF/services/javax.json.spi.JsonProvider. So rather than asking everybody to do that, it would be even nicer to embed the JSR353 Reference Implementation (a mere 64Kb) and let advanced users override the parser by writing the services file. I know that adding external dependencies is not everybody's favorite idea here, but I really believe it definitely help (most) users and would allow us to ship a driver that would work out-of-the-box with JSON.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

On 28/06/15 11:51, Markus KARG wrote:
It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.
    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).

I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.
    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,


    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:
I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of
JDBC
5.0 provides a standard API for dealing with JSON inside of the driver or
at
least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for
JSON
and / or XML.
      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro





--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc





Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 28/06/15 22:54, Sehrope Sarkuni wrote:
On Sun, Jun 28, 2015 at 4:35 PM, Dave Cramer <pg@fastcrypt.com> wrote:
On 28 June 2015 at 16:32, Christopher BROWN <brown@reflexe.fr> wrote:

Embedding the API will cause classloader conflicts for those who already have the API in their classpath.  Same goes for embedding the reference implementation.

The API, or the implementation ???  

Both. Embedding dependencies is mildly convenient for someone getting started with something but royally inconvenient for someone with a complicated environment. Dependency management systems like Maven handle this pretty well but even they can't deal with situations where classes are embedded in a dependency (ex: javax.json.* embedded within the PG JDBC driver).
    Sehrope, what do you mean by Maven can't deal with embedded classes in the dependencies? It very well can :) The only problem would be, as stated in a previous email, if different versions of the same class would collide on transitive dependencies. But at least for the JSR353 IMHO that's extremely unlikely --and not worth to engineer around it.

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

so how do we make it "Just Work" ?
 
I don't think there's a way to do this without breaking backward compatibility.

You definitely can't rely on specific dependencies on the classpath that legacy users will not have. Bundling the dependencies doesn't work either as it'd clobber existing ones on the classpath. Making things dynamic/pluggable may be option for internal implementations but it breaks anything with the required dependencies in the method signature. That means we could have code that dynamically picks a JSON parser and uses it internally, but we can't have a getJsonValue() method on a public PGResultSet interface as the class wouldn't even load properly on an older JVM[1].

In a lot of ways this is similar to the other thread we had about dropping support for older JVMs. To natively support new features (like a native getJsonValue()) we'd need to specify a min JDK version.

    JSR353 targets 1.6+. So if by older you mean 4 or 5 then yes, it won't be supported.

At this point I think we have to bite bullet and either drop support for older versions (not likely) or offer multiple versions of the driver. The latter could make modern assumptions about the classpath/environment to support new features natively. That could include JsonValue and the new Java 8 date/time types.
+1 to drop support for less than 1.6. Or at least, create different versions (but that is indirectly happening now, with different JDBC API levels). But not adding as of today JsonValue to a method signature because we want to retain compatibility with 1.5 or less is not something our users would understand.

    Regards,

    Álvaro

-- 
Álvaro Hernández Tortosa


-----------
8Kdata

Re: SQLJSON

From
Sehrope Sarkuni
Date:
On Mon, Jun 29, 2015 at 2:30 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
On 28/06/15 22:54, Sehrope Sarkuni wrote:
Both. Embedding dependencies is mildly convenient for someone getting started with something but royally inconvenient for someone with a complicated environment. Dependency management systems like Maven handle this pretty well but even they can't deal with situations where classes are embedded in a dependency (ex: javax.json.* embedded within the PG JDBC driver).
    Sehrope, what do you mean by Maven can't deal with embedded classes in the dependencies? It very well can :) The only problem would be, as stated in a previous email, if different versions of the same class would collide on transitive dependencies. But at least for the JSR353 IMHO that's extremely unlikely --and not worth to engineer around it.

I was referring to dependencies being in the pgjdbc jar itself, not specified as an dependency in the pom. For example if we had a single version of the driver that was self-contained with the class files for the javax.json classes in their as well. Maven can't work around that as the classes would be within the pgjdbc depdendency.

Honestly it's not really a Maven issue at all. There's no real way to work around classpath conflicts like that without unzipping/removing/rezipping the jar. 

Much better to either have them be externally specified (ex: Maven) or just assume they're there and leave it to the user to specify them. If users are manually managing dependencies it's up to them to ensure they're all included.
 

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

so how do we make it "Just Work" ?
 
I don't think there's a way to do this without breaking backward compatibility.

You definitely can't rely on specific dependencies on the classpath that legacy users will not have. Bundling the dependencies doesn't work either as it'd clobber existing ones on the classpath. Making things dynamic/pluggable may be option for internal implementations but it breaks anything with the required dependencies in the method signature. That means we could have code that dynamically picks a JSON parser and uses it internally, but we can't have a getJsonValue() method on a public PGResultSet interface as the class wouldn't even load properly on an older JVM[1].

In a lot of ways this is similar to the other thread we had about dropping support for older JVMs. To natively support new features (like a native getJsonValue()) we'd need to specify a min JDK version.

    JSR353 targets 1.6+. So if by older you mean 4 or 5 then yes, it won't be supported.

Oh that's nice then. I haven't read through the spec for it but based on the release date assumed it was 1.7+. This might make things a bit easier.

At this point I think we have to bite bullet and either drop support for older versions (not likely) or offer multiple versions of the driver. The latter could make modern assumptions about the classpath/environment to support new features natively. That could include JsonValue and the new Java 8 date/time types.
+1 to drop support for less than 1.6. Or at least, create different versions (but that is indirectly happening now, with different JDBC API levels). But not adding as of today JsonValue to a method signature because we want to retain compatibility with 1.5 or less is not something our users would understand.

I'm really liking the separate version approach. We're gonna have to do it anyway for JDBC 4.2 ... might as well support PG extensions in the same way.
 
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Re: SQLJSON

From
Dave Cramer
Date:
So assuming the reference implementation allows us to do this, we can change the namespace of the RI and include it in our jar. Apparently Hibernate used this approach 

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 29 June 2015 at 06:28, Sehrope Sarkuni <sehrope@jackdb.com> wrote:
On Mon, Jun 29, 2015 at 2:30 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
On 28/06/15 22:54, Sehrope Sarkuni wrote:
Both. Embedding dependencies is mildly convenient for someone getting started with something but royally inconvenient for someone with a complicated environment. Dependency management systems like Maven handle this pretty well but even they can't deal with situations where classes are embedded in a dependency (ex: javax.json.* embedded within the PG JDBC driver).
    Sehrope, what do you mean by Maven can't deal with embedded classes in the dependencies? It very well can :) The only problem would be, as stated in a previous email, if different versions of the same class would collide on transitive dependencies. But at least for the JSR353 IMHO that's extremely unlikely --and not worth to engineer around it.

I was referring to dependencies being in the pgjdbc jar itself, not specified as an dependency in the pom. For example if we had a single version of the driver that was self-contained with the class files for the javax.json classes in their as well. Maven can't work around that as the classes would be within the pgjdbc depdendency.

Honestly it's not really a Maven issue at all. There's no real way to work around classpath conflicts like that without unzipping/removing/rezipping the jar. 

Much better to either have them be externally specified (ex: Maven) or just assume they're there and leave it to the user to specify them. If users are manually managing dependencies it's up to them to ensure they're all included.
 

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

so how do we make it "Just Work" ?
 
I don't think there's a way to do this without breaking backward compatibility.

You definitely can't rely on specific dependencies on the classpath that legacy users will not have. Bundling the dependencies doesn't work either as it'd clobber existing ones on the classpath. Making things dynamic/pluggable may be option for internal implementations but it breaks anything with the required dependencies in the method signature. That means we could have code that dynamically picks a JSON parser and uses it internally, but we can't have a getJsonValue() method on a public PGResultSet interface as the class wouldn't even load properly on an older JVM[1].

In a lot of ways this is similar to the other thread we had about dropping support for older JVMs. To natively support new features (like a native getJsonValue()) we'd need to specify a min JDK version.

    JSR353 targets 1.6+. So if by older you mean 4 or 5 then yes, it won't be supported.

Oh that's nice then. I haven't read through the spec for it but based on the release date assumed it was 1.7+. This might make things a bit easier.

At this point I think we have to bite bullet and either drop support for older versions (not likely) or offer multiple versions of the driver. The latter could make modern assumptions about the classpath/environment to support new features natively. That could include JsonValue and the new Java 8 date/time types.
+1 to drop support for less than 1.6. Or at least, create different versions (but that is indirectly happening now, with different JDBC API levels). But not adding as of today JsonValue to a method signature because we want to retain compatibility with 1.5 or less is not something our users would understand.

I'm really liking the separate version approach. We're gonna have to do it anyway for JDBC 4.2 ... might as well support PG extensions in the same way.
 
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/


Re: SQLJSON

From
"Markus KARG"
Date:

As I am not a PostgreSQL official, I cannot perform such a poll, as it certainly has to be conducted using the PostgreSQL homepage and mailing list.

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Sonntag, 28. Juni 2015 17:27
To: Markus KARG
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Hi Markus,

 

While I think that would be a wonderful idea, my experience in the last 15 years of working with this driver is that nobody will respond to your question. That being said feel free to take a poll, or otherwise solicit responses. 


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 28 June 2015 at 11:13, Markus KARG <markus@headcrashing.eu> wrote:

Wouldn't it be more clever to ask the pgjdbc *users* what *they* expect to get instead of solely relying upon us three? As the discussion turned out, we have fundamentally diverse opinions, and I doubt that we will do it any right if we just do "something" simply for the benefit of writing "JSON support" on our driver.

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Sonntag, 28. Juni 2015 15:56
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

So I think we should support JSR 353 at the very least Whether we extend the result set or not we can at a minimum return a JsonValue  from getObject

 

I agree with Alvaro, 99% of the users would just like a JsonValue returned. It would be nice if we could design this so more advanced users could plug in their parser of choice.


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 28 June 2015 at 06:00, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


On 28/06/15 11:51, Markus KARG wrote:

It is not *us* who let the JSON users down, it is the PostgreSQL protocol
guys who did not add any useful support for JSON. A driver is not a
compensation for missing product features, it is just a thin wrapper around
the base product's features.

    To have proper JSON support at the protocol level (something which I'd love to have) only translates to more performance, no more functionality. So is a nice-to-have, not a must-to-have (as is supporting PostgreSQL's json data types).


I mean, what happens if the application shall work with a different product?
If you rely on non-JDBC-features, you're screwed. So a profession
application using JSON should ALWAYS come with JSR 253 anyways.

    We have had to extend JDBC in several ways in the past. We should do it again, now, in the best possible manner (getObject, PGResultSet, whatever). And then, if JDBC adds support in the future, retrofit into it. But not wait until then, because we don't even know if that would even happen.

    Cheers,



    Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata


-----Original Message-----
From: pgsql-jdbc-owner@postgresql.org
[mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández
Tortosa
Sent: Sonntag, 28. Juni 2015 11:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON


On 28/06/15 11:17, Markus KARG wrote:

I do not see the benefit of that effort, as getting JSON as a LONG VARCHAR
and then parsing it on behalf of the application is pretty simple and
straightforward. My vote would be to not do anything until JDBC 4.3 of

JDBC

5.0 provides a standard API for dealing with JSON inside of the driver or

at

least PostgreSQL 9.5 or PostgreSQL 10 provides a streaming protocol for

JSON

and / or XML.

      Don't do anything?

      And let Java PostgreSQL users down, without a (driver, supported)
means of getting JSON out of their database? So we make the "marketing"
that 9.4 is all about jsonb and the NoSQL replacement yet you cannot do
JSON with Java?

      Really?

      User's don't care about extreme performance. Users care about easy
of use and decent set of features. Adding JSON support, even thought
it's not the most performant one is something we should be doing as
quickly as possible.

      Regards,

      Álvaro




--

Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 

 

Re: SQLJSON

From
"Markus KARG"
Date:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject. I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.

 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.

 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.



  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.




  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 







--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)




 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)




 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)



 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp




 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.

I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....

 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.


    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.



  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.




  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 







--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)




 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)




 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)



 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp




 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 29/06/15 15:54, Dave Cramer wrote:
So assuming the reference implementation allows us to do this, we can change the namespace of the RI and include it in our jar. Apparently Hibernate used this approach

    This makes a lot of sense, really good idea.

    +1

    Álvaro
-- 
Álvaro Hernández Tortosa


-----------
8Kdata



Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 29 June 2015 at 06:28, Sehrope Sarkuni <sehrope@jackdb.com> wrote:
On Mon, Jun 29, 2015 at 2:30 AM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
On 28/06/15 22:54, Sehrope Sarkuni wrote:
Both. Embedding dependencies is mildly convenient for someone getting started with something but royally inconvenient for someone with a complicated environment. Dependency management systems like Maven handle this pretty well but even they can't deal with situations where classes are embedded in a dependency (ex: javax.json.* embedded within the PG JDBC driver).
    Sehrope, what do you mean by Maven can't deal with embedded classes in the dependencies? It very well can :) The only problem would be, as stated in a previous email, if different versions of the same class would collide on transitive dependencies. But at least for the JSR353 IMHO that's extremely unlikely --and not worth to engineer around it.

I was referring to dependencies being in the pgjdbc jar itself, not specified as an dependency in the pom. For example if we had a single version of the driver that was self-contained with the class files for the javax.json classes in their as well. Maven can't work around that as the classes would be within the pgjdbc depdendency.

Honestly it's not really a Maven issue at all. There's no real way to work around classpath conflicts like that without unzipping/removing/rezipping the jar. 

Much better to either have them be externally specified (ex: Maven) or just assume they're there and leave it to the user to specify them. If users are manually managing dependencies it's up to them to ensure they're all included.
 

The service loader API can be problematic for OSGi users, as it isn't very helpful for hot reloading of classes.  The PostgreSQL JDBC driver currently works well in such environments, it would be unfortunate to lose that advantage through an attempt to help out another category of users.

This shouldn't be the only way of selecting an implementation, and bundling a given version of the API + RI shouldn't be the only build option. I'm certainly not against making this Just Work, but here there's a possibility that all this extra stuff could actually cause things to break .

so how do we make it "Just Work" ?
 
I don't think there's a way to do this without breaking backward compatibility.

You definitely can't rely on specific dependencies on the classpath that legacy users will not have. Bundling the dependencies doesn't work either as it'd clobber existing ones on the classpath. Making things dynamic/pluggable may be option for internal implementations but it breaks anything with the required dependencies in the method signature. That means we could have code that dynamically picks a JSON parser and uses it internally, but we can't have a getJsonValue() method on a public PGResultSet interface as the class wouldn't even load properly on an older JVM[1].

In a lot of ways this is similar to the other thread we had about dropping support for older JVMs. To natively support new features (like a native getJsonValue()) we'd need to specify a min JDK version.

    JSR353 targets 1.6+. So if by older you mean 4 or 5 then yes, it won't be supported.

Oh that's nice then. I haven't read through the spec for it but based on the release date assumed it was 1.7+. This might make things a bit easier.

At this point I think we have to bite bullet and either drop support for older versions (not likely) or offer multiple versions of the driver. The latter could make modern assumptions about the classpath/environment to support new features natively. That could include JsonValue and the new Java 8 date/time types.
+1 to drop support for less than 1.6. Or at least, create different versions (but that is indirectly happening now, with different JDBC API levels). But not adding as of today JsonValue to a method signature because we want to retain compatibility with 1.5 or less is not something our users would understand.

I'm really liking the separate version approach. We're gonna have to do it anyway for JDBC 4.2 ... might as well support PG extensions in the same way.
 
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/



Re: SQLJSON

From
Dave Cramer
Date:
Markus,

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.


Kind Regards,

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.

I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....

 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.



    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.



  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.




  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 







--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)




 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)




 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)



 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp




 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 



Re: SQLJSON

From
"Markus KARG"
Date:

You're overdramatizing things and I did not want to offend you (sorry if you feel that way, it is not on purpose). Actually I meant it literally when I said "just to make you happy", as I do not see any other reason to provide a solution, as it does not bring ANY benefit to the user BUT implies future problems as I and others already explained. I do not know on what basis you say you're speaking for "all" users (if you are a PostgreSQL official in some way or have done a poll please just tell me), and I do not see what harm it does to the good work done INSIDE of PostgreSQL wrt JSON if pgjdbc does not provide JsonValue, as I explained (two times meanwhile) already that you can use that great work with the EXISTING driver. Also I do not see in what way a user is forced to use MongoDB just because we like to wait with a driver change until the JDBC working group decided to add JSON. I do not see how you "help" users just by moving the parser from the application into the driver, actually.

 

Sorry I totally do not see your complete point in all what your write. Maybe something else can chime in and explaint so I see Alvaro's point?

 

Thanks

Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Montag, 29. Juni 2015 21:02
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.


    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.




  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.





  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro






-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 








--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)





 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)





 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)




 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp





 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

Re: SQLJSON

From
"Markus KARG"
Date:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 29/06/15 21:41, Markus KARG wrote:

You're overdramatizing things and I did not want to offend you (sorry if you feel that way, it is not on purpose). Actually I meant it literally when I said "just to make you happy",


    Markus, I don't want to keep on engaging in non-productive debates, but again I think you should moderate your language. Saying now I am overdramatizing is a realization of this.

as I do not see any other reason to provide a solution, as it does not bring ANY benefit to the user BUT implies future problems as I and others already explained.

    Maybe I am not explaining myself, but that you doesn't understand what I say doesn't mean you have to reduce my goals to the realm of myself, specially if I have very clearly and loudly stated that I want something for all users, or most users. So ignoring my references is not a nice summary of my speech.

I do not know on what basis you say you're speaking for "all" users (if you are a PostgreSQL official in some way or have done a poll please just tell me),


    It's an unofficial, belief-based approach. But follows a logical perspective, and is also based on true conversations with many users, with which I am in permanent contact. And everybody (mostly) wants to use JSON in a natural way from the drivers.

and I do not see what harm it does to the good work done INSIDE of PostgreSQL wrt JSON if pgjdbc does not provide JsonValue, as I explained (two times meanwhile) already that you can use that great work with the EXISTING driver.


    Not without difficulties. Not without a great user-experience.

Also I do not see in what way a user is forced to use MongoDB just because we like to wait with a driver change until the JDBC working group decided to add JSON. I do not see how you "help" users just by moving the parser from the application into the driver, actually.

    You care a lot about performance. That's great. But unfortunately, user-friendliness, and featureset, are mostly always preferred by users over extreme performance. And very fast products that nobody use end up going away.

    All I want is to compete, with a better product, but with at least the same user-friendliness as other, possibly competing alternatives in this area. And this requires the driver to "support JSON", whatever that means (I think what it means, but I may be wrong).

 

Sorry I totally do not see your complete point in all what your write. Maybe something else can chime in and explaint so I see Alvaro's point?


    I think Dave did a way better job explaining what I mean.

    Having that said, I insist that you will have me by your side to improve JSON at the protocol level and include any optimizations there. Just ping me back by then.

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata


 

Thanks

Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Montag, 29. Juni 2015 21:02
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.


    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.




  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.





  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro






-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 








--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)





 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)





 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)




 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp





 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 


Re: SQLJSON

From
Dave Cramer
Date:
On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?


Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.  

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)


Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)


As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.

Regards,

Dave 

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 


Re: SQLJSON

From
Christopher BROWN
Date:
Hello,

In summary, is this what is planned ?
  • Bundle the JSONP API with the driver (specific version)
  • Bundle the JSONP RI with the driver (specific version, with modified package names)
  • Override getObject() to return a "JSON DOM" (buffering data once, then reconstructing it again as an object model)
Is that correct?  Could you remind me what does getObject() currently return for JSONB (I don't have a test case to hand)?

Thanks,
Christopher


On 29 June 2015 at 22:58, Dave Cramer <pg@fastcrypt.com> wrote:
On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?


Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.  

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)


Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)


As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.

Regards,

Dave 

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 



Re: SQLJSON

From
Dave Cramer
Date:



On 29 June 2015 at 17:19, Christopher BROWN <brown@reflexe.fr> wrote:
Hello,

In summary, is this what is planned ?
  • Bundle the JSONP API with the driver (specific version)
  • Bundle the JSONP RI with the driver (specific version, with modified package names)
  • Override getObject() to return a "JSON DOM" (buffering data once, then reconstructing it again as an object model)
Is that correct?  Could you remind me what does getObject() currently return for JSONB (I don't have a test case to hand)?

That is the current proposal, yes. I would expect getObject to return a string at this point. But I haven't tested it either

Dave 

Thanks,
Christopher


On 29 June 2015 at 22:58, Dave Cramer <pg@fastcrypt.com> wrote:
On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?


Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.  

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)


Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)


As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.

Regards,

Dave 

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 




Re: SQLJSON

From
Steven Schlansker
Date:
I apologize in advance if I've missed something, the volume of emails in this thread is getting ... a bit overwhelming.
My understanding of the situation is:

* Without JDBC5, we cannot expect our solution to be portable
* Without PG backend / protocol improvements, we cannot have our solution be faster or more efficient
* Without bloating the driver (and locking ourselves in to not receiving upstream bug fixes without re-rolling the
driver)or requiring external dependencies, we cannot have a nicer API to return JSON (and even if we do, see #1) 

Particularly in the case of using SQLJSON as a cursor-like construct to partially retrieve JSON values (did I
understandthat correctly?), it feels like we are rushing to implement something that cannot possibly be implemented
efficientlytoday, and if it does eventually get server / protocol support, our API is just as likely to be a total
mismatchas it is to mesh well. 

I would also like to point out that adding new methods to PGConnection is great, but as soon as you introduce a
connectionpool or other type of facade (which is, to my understanding, by far the common case) using any non-interface
methodsbecomes extremely confusing.  At the very best you have to deal with "unwrapping" any potentially wrapped
objects(Connection, ResultSet, Statement, etc) and worse you may find your pool does not bother to implement the
Wrapperinterface since it did not even exist until Java 6 and these tools move extremely slowly.  And it all breaks
whenyou switch connection pools and they have a different way of doing the wrapping... 

I wonder if it would not be more prudent to wait until JDBC has an API to argue about, or Postgres protocol supports at
leastone of the proposed features, rather than rushing to bake our own at the that we know has very little improvement
overwhat you can do out of the box today.  We are trying to build a feature from the middle out, rather than letting it
bedriven either top-down or bottom-up. 

I hope I haven't misunderstood anything here, I'm just having a very hard time figuring out the actual use case we are
tryingto solve from the large mash of emails.  To be clear, I say this as someone who has used the Postgres JSON
supportrelatively extensively, and would love to see improvements.  Maybe one of the advocates here could provide some
"pseudo-code"use cases?  A "story" for the proposal, if you will?  Sell me, make me wish for this feature :) 

To me the litmus test would be how much you can improve over the naive "end user ties Jackson and JDBC together"
approach,namely things such as 
rs = ps.execute("SELECT json_field FROM table WHERE condition = 42");
domainObject = mapper.readValue(rs.getBytes("json_field"), DomainObject.class);

either in terms of
* Code cleanliness that is more fundamental than a syntactic sugar helper method
 - For example a getJsonValue(rs, "json_field", DomainObject.class) shorthand
* Performance
 - Processing time
 - Memory usage
* Versatility
 - Not sure what people are missing here?  Except the SQLJSON cursor which we cannot possibly implement efficiently
today

Anyway, I'm really not trying to be a wet blanket here, I just want to make sure we all have a clear idea of what
problemswe are and are not solving by this proposed feature.  Don't want to stretch our already limited pg-jdbc
developmentresource even more thinly! 

On Jun 29, 2015, at 1:58 PM, Dave Cramer <pg@fastcrypt.com> wrote:

> On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:
> Dave,
>
>
>
> I understand that people expect that getObject does something reasonable - but the case why this does not happen is
becausePostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have
missedthat, actually, then please just tell me)? 
>
>
>
> Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If
youthink this mailing list is contentious you might want to have a look at hackers. To answer your question though
thereis no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is
whatit is.   
>
>
> But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has
toreturn FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return
butnot "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to
nobodyat all. Anyways, you're the boss, and I trust in you to do the right choice. :-) 
>
>
> Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer
here.Howeverin the absence of the spec, I'm sure whatever we do will be a compromise. 
>
>
>
> Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are
professionalcombustion engine engineers in our main job. Our whole life is dedicated to the difference between a
vanillaBMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your
attitudethat you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer,
wesimply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and
consumptionjust because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and
safetyand cost of production and unassembly for recycling, and usefulness and maintainability and testability of the
car,too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and
performanceof the end product is not what you like to achieve and you like to simply replace the pipe with a chromed
one,then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-) 
>
>
> As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to
keepthe majority of the users in mind. We also need to make sure in our quest for performance that we don't break
anything.
>
> Regards,
>
> Dave
>
>
>
> Have fun
>
> -Markus
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
> Sent: Montag, 29. Juni 2015 21:23
> To: Álvaro Hernández Tortosa
> Cc: List
> Subject: Re: [JDBC] SQLJSON
>
>
>
> Markus,
>
>
>
> I really value your recent input to the driver and would like it to continue but we need to keep things in
perspective.The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with
9'sif you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver
todo something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject
andthat object is a json value they expect something magical to happen. We return an object they can use. 
>
>
>
> The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a
metaphorif you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is
nothingnew about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes
whichconnect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect
thecar to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to
connectthe engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly
thanklessjob. 
>
>
>
>
>
> Kind Regards,
>
>
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
>
>
> On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>     Markus:
>
> On 29/06/15 18:26, Markus KARG wrote:
>
> Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical
realitythat an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be
ableto process JsonObject. 
>
>
>     It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of
course. 
>
>
>
> I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver
supports.
>
>
>     Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON
justbecause you fear of an absolutely unrealistic class path? Really? Please.... 
>
>
>
>
>
> The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that
oursolution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc
contributorsjust for the sake to make you happy, without providing you any measurable benefit. 
>
>
>     Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you
cannot,please abandon this otherwise respectful and enriching debate. 
>
>     I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission,
ofcourse). 
>
>     Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or
somefears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our
users,not help them (with your help) to go to MongoDB. 
>
>
>
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> -Markus
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 21:06
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 17:09, Markus KARG wrote:
>
> If your application does not want to deal with JSON data then don't request it from the database, but use
http://www.postgresql.org/docs/9.4/static/functions-json.htmlin your SELECT statement. Performing a JSON extraction
functionnatively on the server and simply use the existing data types in your application not only reliefs us from
dealingwith JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work
bydefault already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I
stilldo not understand you exact scenario. 
>
>
>     Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.
>
>     All I'm saying is:
>
> - Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the
resultof an expression which evaluates to any of those. 
>
> - Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to
loadany other code, unless they want to override the default JSON parser. 
>
>     From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that
itworks by default and that you can query JSON. Is this that weird to ask? 
>
>     I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks
ofmy proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument
againstthis. 
>
>     Thanks,
>
>     Álvaro
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 11:57
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 11:49, Markus KARG wrote:
>
> You miss one essential point against bundling: An application author who wants to process JsonObject must have that
classon his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So
wheredoes he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply
notpgjdbc's problem as in that scenario no JDBC is used at all. 
>
>     I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as
non-JSONdata. It all depends on your domain Objects. 
>
>     I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding
a64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user. 
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
> Sent: Sonntag, 28. Juni 2015 11:41
> To: Christopher BROWN
> Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 09:34, Christopher BROWN wrote:
>
> Hello,
>
>
>
> Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting
unreadable.
>
>     • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no
definitiveform (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it
makesno sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't
actuallyimplement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility
API)it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might
chooseJackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause
meclassloader issues as I often update to the latest version of Jackson). 
>     Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not
implementJSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it
doesnot need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of
dependenciesto make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size.
Iwould not be scared however to see a 2Mb jdbc jar file. 
>
>     However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the
Jarfile. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't
considerthis offensive. 
>
>     In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use
Jackson?Go for it. The rest of the world would have something running out-of-the-box. 
>
>
>
>     • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.
It'stherefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to
theclasspath. 
>     • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for
others,you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their
application).
>     Technically, it's possible. But from a user perspective, not bundling an implementation means:
>
> - Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
> - Google that. Find that you need to add the dependency and very likely create a SPI file
(META-INF/services/javax.json.spi.JsonProvider).
> - Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my
opinion,but might be user's opinion). 
> - Google again to see what JSR353-compliant implementations are out there.
> - Blame PostgreSQL again for not making this choice for you.
> - Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the
"quality"of that wrapper and consider whether that would be "supported" with PostgreSQL driver. 
> - Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the
ReferenceImplementation (there are not many JSR353 implementations, after all). 
> - Choose without knowing which JSON parser is best.
> - Bundle the dependency, check now it works. Clean your sweat.
> - Wonder why not to choose MongoDB next time, it works out of the box.
>
>     Not funny.
>
>     The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to
makeanother choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the
bundledRI). You have the best of both worlds. 
>
>     Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a
fileand "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process. 
>
>
>
>
>     • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add
streamingsupport that reads from already fully-read resources... because in that way, if the protocol improves in the
future,client code using the streaming API will benefit (with no changes to client code) in the future. 
>     JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.
>
>     Best regards,
>
>     Álvaro
>
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> --
>
> Christopher
>
>
>
>
>
> On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>
> On 28/06/15 00:55, Sehrope Sarkuni wrote:
>
> On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>     Hi Sehrope!
>
>
>
> Hi Álvaro! :D
>
>
>
>     To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included
one,for another one. Just picking a sensible default (Jackson, for instance) is probably good enough. 
>
>
>
> I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in
thatit lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we
couldhave something like: <T> T getJsonAsType(String, Class<T> clazz) 
>
>
>     That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it
returnsyou the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue) 
>
>
>
>
>
>
> I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still
needto figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?). 
>
>
>     JSR353 is targeted for JavaSE 6 :)
>
>
>
>
>
>
>
> #2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON
isa bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an
objectbut "foobar" is valid JSON as well. This leads us to #3... 
>
>
>     The object type to return has to be IMHO JsonObject:
http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
>
>
>
> Not always though. All these are valid JSON too:
>
>
>
> => SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
>
>  num | string |   arr   |      obj
>
> -----+--------+---------+---------------
>
>  1   | "test" | [1,2,3] | {"foo":"bar"}
>
> (1 row)
>
>
>
> We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be
usingthe object type though (and maybe the array). 
>
>
>     You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including
objectsand arrays. So it would be enough just with JsonValue getJsonValue(....) 
>
>
>
>
>
>
> #3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be
returnedvia getObject(...). 
>
> An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG
extensions.They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to
retrieveJSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably
mostpeople using json/jsonb types know the top level type of what they're storing. 
>
>
>     Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the
methods,if they simply return JsonObject, you already have a full API there to parse and extract and process. So
anythingthat returns a JsonObject from a column (identifier or #) would be enough for me. 
>
>
>
> For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything
beyondthat would be a full on transformation and would be very application specific. 
>
>
>     Yepp
>
>
>
>
>
>
> For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing
withjson/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of
theircode. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert
toour "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java
typeswould make this more convenient. 
>
> The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using
ithas to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out
ofthe way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input
stream(it'll be buffered in a byte array), but at least it won't be copied twice. 
>
>
>     As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't
expecthowever big performance wins as JSON is sent as a String over the wire... 
>
>
>
> The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes
=>JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too. 
>
>
>     You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid
convertingto String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out
layingin user's realm, beyond driver's responsibility). 
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
>
>
>
>
>
>
>



Re: SQLJSON

From
Steven Schlansker
Date:
For what it's worth, I think bundling the JSONP API with the driver is a bad idea.  As a Maven user, every time I come
acrossa jar file where someone has "helpfully" repackaged some core Java piece into their jar, I curse them repeatedly
andthen have to try to figure out how either to split it out or somehow otherwise reconcile that they have packaged
e.g.javax.ws.rs 2.0b01 and I need to use a 2.1 feature. 

At the very least, if it must be bundled, please make it a separate jar than what Maven users get.  (To be clear I'm
totallyfine with a Maven dependency on whatever JSR spec jar is needed, I only object to bundling it) 

I am also moderately against bundling the RI, although I can at least see how it is a convenience for "get off the
groundrunning" use cases.  In particular I have multiple times run into bugs that have been baked into unrelated
librariesdue to this issue (most recently, a cglib bug preventing the library working on Java 8, with cglib shaded into
anunrelated library) 

Be sure to expect that you will now have to do driver releases for no reason other than to bundle a newer RI with a
criticalbug fix.  There are always more bugs to find, no matter how stable the software is... and by bundling it you
havemade that your problem, rather than "just get a newer version from upstream". 

I would not be opposed to having "Driver rollup" bundles like a "jdbc + jsonp-spec + jsonp-ri" jar.  I just don't want
thatcrap in my Maven build, because it *will* cause problems for moderate-level and above users :) 

On Jun 29, 2015, at 2:21 PM, Dave Cramer <pg@fastcrypt.com> wrote:

>
>
>
> On 29 June 2015 at 17:19, Christopher BROWN <brown@reflexe.fr> wrote:
> Hello,
>
> In summary, is this what is planned ?
>     • Bundle the JSONP API with the driver (specific version)
>     • Bundle the JSONP RI with the driver (specific version, with modified package names)
>     • Override getObject() to return a "JSON DOM" (buffering data once, then reconstructing it again as an object
model)
> Is that correct?  Could you remind me what does getObject() currently return for JSONB (I don't have a test case to
hand)?
>
> That is the current proposal, yes. I would expect getObject to return a string at this point. But I haven't tested it
either
>
> Dave
>
> Thanks,
> Christopher
>
>
> On 29 June 2015 at 22:58, Dave Cramer <pg@fastcrypt.com> wrote:
> On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:
> Dave,
>
>
>
> I understand that people expect that getObject does something reasonable - but the case why this does not happen is
becausePostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have
missedthat, actually, then please just tell me)? 
>
>
>
> Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If
youthink this mailing list is contentious you might want to have a look at hackers. To answer your question though
thereis no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is
whatit is.   
>
>
> But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has
toreturn FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return
butnot "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to
nobodyat all. Anyways, you're the boss, and I trust in you to do the right choice. :-) 
>
>
> Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer
here.Howeverin the absence of the spec, I'm sure whatever we do will be a compromise. 
>
>
>
> Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are
professionalcombustion engine engineers in our main job. Our whole life is dedicated to the difference between a
vanillaBMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your
attitudethat you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer,
wesimply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and
consumptionjust because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and
safetyand cost of production and unassembly for recycling, and usefulness and maintainability and testability of the
car,too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and
performanceof the end product is not what you like to achieve and you like to simply replace the pipe with a chromed
one,then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-) 
>
>
> As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to
keepthe majority of the users in mind. We also need to make sure in our quest for performance that we don't break
anything.
>
> Regards,
>
> Dave
>
>
>
> Have fun
>
> -Markus
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
> Sent: Montag, 29. Juni 2015 21:23
> To: Álvaro Hernández Tortosa
> Cc: List
> Subject: Re: [JDBC] SQLJSON
>
>
>
> Markus,
>
>
>
> I really value your recent input to the driver and would like it to continue but we need to keep things in
perspective.The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with
9'sif you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver
todo something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject
andthat object is a json value they expect something magical to happen. We return an object they can use. 
>
>
>
> The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a
metaphorif you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is
nothingnew about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes
whichconnect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect
thecar to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to
connectthe engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly
thanklessjob. 
>
>
>
>
>
> Kind Regards,
>
>
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
>
>
> On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>     Markus:
>
> On 29/06/15 18:26, Markus KARG wrote:
>
> Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical
realitythat an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be
ableto process JsonObject. 
>
>
>     It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of
course. 
>
>
>
> I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver
supports.
>
>
>     Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON
justbecause you fear of an absolutely unrealistic class path? Really? Please.... 
>
>
>
>
>
> The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that
oursolution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc
contributorsjust for the sake to make you happy, without providing you any measurable benefit. 
>
>
>     Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you
cannot,please abandon this otherwise respectful and enriching debate. 
>
>     I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission,
ofcourse). 
>
>     Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or
somefears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our
users,not help them (with your help) to go to MongoDB. 
>
>
>
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> -Markus
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 21:06
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 17:09, Markus KARG wrote:
>
> If your application does not want to deal with JSON data then don't request it from the database, but use
http://www.postgresql.org/docs/9.4/static/functions-json.htmlin your SELECT statement. Performing a JSON extraction
functionnatively on the server and simply use the existing data types in your application not only reliefs us from
dealingwith JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work
bydefault already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I
stilldo not understand you exact scenario. 
>
>
>     Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.
>
>     All I'm saying is:
>
> - Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the
resultof an expression which evaluates to any of those. 
>
> - Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to
loadany other code, unless they want to override the default JSON parser. 
>
>     From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that
itworks by default and that you can query JSON. Is this that weird to ask? 
>
>     I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks
ofmy proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument
againstthis. 
>
>     Thanks,
>
>     Álvaro
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 11:57
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 11:49, Markus KARG wrote:
>
> You miss one essential point against bundling: An application author who wants to process JsonObject must have that
classon his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So
wheredoes he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply
notpgjdbc's problem as in that scenario no JDBC is used at all. 
>
>     I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as
non-JSONdata. It all depends on your domain Objects. 
>
>     I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding
a64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user. 
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
> Sent: Sonntag, 28. Juni 2015 11:41
> To: Christopher BROWN
> Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 09:34, Christopher BROWN wrote:
>
> Hello,
>
>
>
> Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting
unreadable.
>
>     • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no
definitiveform (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it
makesno sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't
actuallyimplement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility
API)it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might
chooseJackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause
meclassloader issues as I often update to the latest version of Jackson). 
>     Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not
implementJSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it
doesnot need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of
dependenciesto make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size.
Iwould not be scared however to see a 2Mb jdbc jar file. 
>
>     However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the
Jarfile. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't
considerthis offensive. 
>
>     In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use
Jackson?Go for it. The rest of the world would have something running out-of-the-box. 
>
>
>
>     • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.
It'stherefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to
theclasspath. 
>     • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for
others,you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their
application).
>     Technically, it's possible. But from a user perspective, not bundling an implementation means:
>
> - Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
> - Google that. Find that you need to add the dependency and very likely create a SPI file
(META-INF/services/javax.json.spi.JsonProvider).
> - Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my
opinion,but might be user's opinion). 
> - Google again to see what JSR353-compliant implementations are out there.
> - Blame PostgreSQL again for not making this choice for you.
> - Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the
"quality"of that wrapper and consider whether that would be "supported" with PostgreSQL driver. 
> - Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the
ReferenceImplementation (there are not many JSR353 implementations, after all). 
> - Choose without knowing which JSON parser is best.
> - Bundle the dependency, check now it works. Clean your sweat.
> - Wonder why not to choose MongoDB next time, it works out of the box.
>
>     Not funny.
>
>     The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to
makeanother choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the
bundledRI). You have the best of both worlds. 
>
>     Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a
fileand "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process. 
>
>
>
>
>     • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add
streamingsupport that reads from already fully-read resources... because in that way, if the protocol improves in the
future,client code using the streaming API will benefit (with no changes to client code) in the future. 
>     JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.
>
>     Best regards,
>
>     Álvaro
>
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> --
>
> Christopher
>
>
>
>
>
> On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>
> On 28/06/15 00:55, Sehrope Sarkuni wrote:
>
> On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>     Hi Sehrope!
>
>
>
> Hi Álvaro! :D
>
>
>
>     To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included
one,for another one. Just picking a sensible default (Jackson, for instance) is probably good enough. 
>
>
>
> I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in
thatit lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we
couldhave something like: <T> T getJsonAsType(String, Class<T> clazz) 
>
>
>     That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it
returnsyou the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue) 
>
>
>
>
>
>
> I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still
needto figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?). 
>
>
>     JSR353 is targeted for JavaSE 6 :)
>
>
>
>
>
>
>
> #2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON
isa bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an
objectbut "foobar" is valid JSON as well. This leads us to #3... 
>
>
>     The object type to return has to be IMHO JsonObject:
http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
>
>
>
> Not always though. All these are valid JSON too:
>
>
>
> => SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
>
>  num | string |   arr   |      obj
>
> -----+--------+---------+---------------
>
>  1   | "test" | [1,2,3] | {"foo":"bar"}
>
> (1 row)
>
>
>
> We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be
usingthe object type though (and maybe the array). 
>
>
>     You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including
objectsand arrays. So it would be enough just with JsonValue getJsonValue(....) 
>
>
>
>
>
>
> #3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be
returnedvia getObject(...). 
>
> An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG
extensions.They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to
retrieveJSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably
mostpeople using json/jsonb types know the top level type of what they're storing. 
>
>
>     Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the
methods,if they simply return JsonObject, you already have a full API there to parse and extract and process. So
anythingthat returns a JsonObject from a column (identifier or #) would be enough for me. 
>
>
>
> For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything
beyondthat would be a full on transformation and would be very application specific. 
>
>
>     Yepp
>
>
>
>
>
>
> For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing
withjson/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of
theircode. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert
toour "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java
typeswould make this more convenient. 
>
> The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using
ithas to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out
ofthe way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input
stream(it'll be buffered in a byte array), but at least it won't be copied twice. 
>
>
>     As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't
expecthowever big performance wins as JSON is sent as a String over the wire... 
>
>
>
> The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes
=>JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too. 
>
>
>     You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid
convertingto String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out
layingin user's realm, beyond driver's responsibility). 
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



Re: SQLJSON

From
Dave Cramer
Date:
Steven,



On 29 June 2015 at 17:32, Steven Schlansker <stevenschlansker@gmail.com> wrote:
For what it's worth, I think bundling the JSONP API with the driver is a bad idea.  As a Maven user, every time I come across a jar file where someone has "helpfully" repackaged some core Java piece into their jar, I curse them repeatedly and then have to try to figure out how either to split it out or somehow otherwise reconcile that they have packaged e.g. javax.ws.rs 2.0b01 and I need to use a 2.1 feature.

At the very least, if it must be bundled, please make it a separate jar than what Maven users get.  (To be clear I'm totally fine with a Maven dependency on whatever JSR spec jar is needed, I only object to bundling it)

I am also moderately against bundling the RI, although I can at least see how it is a convenience for "get off the ground running" use cases.  In particular I have multiple times run into bugs that have been baked into unrelated libraries due to this issue (most recently, a cglib bug preventing the library working on Java 8, with cglib shaded into an unrelated library)

Be sure to expect that you will now have to do driver releases for no reason other than to bundle a newer RI with a critical bug fix.  There are always more bugs to find, no matter how stable the software is... and by bundling it you have made that your problem, rather than "just get a newer version from upstream".

I would not be opposed to having "Driver rollup" bundles like a "jdbc + jsonp-spec + jsonp-ri" jar.  I just don't want that crap in my Maven build, because it *will* cause problems for moderate-level and above users :)


Yes, that was my intent. Maven users would use the pom, others would get a shaded jar with different names, Which of course brings it's own nightmare.

Dave
 
On Jun 29, 2015, at 2:21 PM, Dave Cramer <pg@fastcrypt.com> wrote:

>
>
>
> On 29 June 2015 at 17:19, Christopher BROWN <brown@reflexe.fr> wrote:
> Hello,
>
> In summary, is this what is planned ?
>       • Bundle the JSONP API with the driver (specific version)
>       • Bundle the JSONP RI with the driver (specific version, with modified package names)
>       • Override getObject() to return a "JSON DOM" (buffering data once, then reconstructing it again as an object model)
> Is that correct?  Could you remind me what does getObject() currently return for JSONB (I don't have a test case to hand)?
>
> That is the current proposal, yes. I would expect getObject to return a string at this point. But I haven't tested it either
>
> Dave
>
> Thanks,
> Christopher
>
>
> On 29 June 2015 at 22:58, Dave Cramer <pg@fastcrypt.com> wrote:
> On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:
> Dave,
>
>
>
> I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?
>
>
>
> Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.
>
>
> But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)
>
>
> Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.
>
>
>
> Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)
>
>
> As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.
>
> Regards,
>
> Dave
>
>
>
> Have fun
>
> -Markus
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
> Sent: Montag, 29. Juni 2015 21:23
> To: Álvaro Hernández Tortosa
> Cc: List
> Subject: Re: [JDBC] SQLJSON
>
>
>
> Markus,
>
>
>
> I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.
>
>
>
> The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.
>
>
>
>
>
> Kind Regards,
>
>
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
>
>
> On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>     Markus:
>
> On 29/06/15 18:26, Markus KARG wrote:
>
> Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.
>
>
>     It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.
>
>
>
> I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.
>
>
>     Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....
>
>
>
>
>
> The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.
>
>
>     Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.
>
>     I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).
>
>     Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.
>
>
>
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> -Markus
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 21:06
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 17:09, Markus KARG wrote:
>
> If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.
>
>
>     Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.
>
>     All I'm saying is:
>
> - Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.
>
> - Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.
>
>     From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?
>
>     I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.
>
>     Thanks,
>
>     Álvaro
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
> From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
> Sent: Sonntag, 28. Juni 2015 11:57
> To: pgsql-jdbc@postgresql.org
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 11:49, Markus KARG wrote:
>
> You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.
>
>     I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.
>
>     I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
> Sent: Sonntag, 28. Juni 2015 11:41
> To: Christopher BROWN
> Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
> Subject: Re: [JDBC] SQLJSON
>
>
>
>
>
> On 28/06/15 09:34, Christopher BROWN wrote:
>
> Hello,
>
>
>
> Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.
>
>       • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).
>     Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.
>
>     However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.
>
>     In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.
>
>
>
>       • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
>       • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).
>     Technically, it's possible. But from a user perspective, not bundling an implementation means:
>
> - Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
> - Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
> - Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
> - Google again to see what JSR353-compliant implementations are out there.
> - Blame PostgreSQL again for not making this choice for you.
> - Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
> - Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
> - Choose without knowing which JSON parser is best.
> - Bundle the dependency, check now it works. Clean your sweat.
> - Wonder why not to choose MongoDB next time, it works out of the box.
>
>     Not funny.
>
>     The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.
>
>     Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.
>
>
>
>
>       • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.
>     JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.
>
>     Best regards,
>
>     Álvaro
>
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
> --
>
> Christopher
>
>
>
>
>
> On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>
>
> On 28/06/15 00:55, Sehrope Sarkuni wrote:
>
> On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:
>
>     Hi Sehrope!
>
>
>
> Hi Álvaro! :D
>
>
>
>     To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.
>
>
>
> I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)
>
>
>     That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)
>
>
>
>
>
>
> I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).
>
>
>     JSR353 is targeted for JavaSE 6 :)
>
>
>
>
>
>
>
> #2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...
>
>
>     The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html
>
>
>
> Not always though. All these are valid JSON too:
>
>
>
> => SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;
>
>  num | string |   arr   |      obj
>
> -----+--------+---------+---------------
>
>  1   | "test" | [1,2,3] | {"foo":"bar"}
>
> (1 row)
>
>
>
> We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).
>
>
>     You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)
>
>
>
>
>
>
> #3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).
>
> An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.
>
>
>     Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.
>
>
>
> For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.
>
>
>     Yepp
>
>
>
>
>
>
> For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.
>
> The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.
>
>
>     As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...
>
>
>
> The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.
>
>
>     You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).
>
>     Regards,
>
>     Álvaro
>
>
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


Re: SQLJSON

From
"Markus KARG"
Date:

Álvaro, please keep calm, I never intended to offend you. I don't know exactly what made you angry, possibly you misunderstood "overdramatizing", which targeted _solely_ at the closing exaggeration that people use MongoDB if we do not instantly provide JDBC support for JSON and that you personally speak for "all" users, but I am really sorry about anything that offended you, and I like to tell you that my intent was solely to work out more hard facts and reduce opinions and beliefs. :-)

 

I did never say you are _only_ talking about yourself. I just said that as the change provides no measureable benefit it "is only good for making you happy". "you" does not mean "Álvaro" in this context, but "all users", even if this is 100% of all users of the driver. At least this is what I meant to say with that line. With "being happy" I meant that the users feel happy, but have no measurable benefit in terms of shorter programs, less configuration trouble, or faster execution, so won't be happy for long. Hence, I did not ignore your references. Sorry if that was unclear.

 

About your beliefs, I also am in contact with really many users, and need to tell you that NONE of them asked for native JSON support in a driver but ALL of them either don't use JSON or use the JSON-functions on the server, and many people talk to me about performance. But as this is simply _my_personal_experience_ of my day-time job (we're an ISV+Consulting, managing 1.200+ enterprises world-wide with different kinds of RDBMS, with PostgreSQL being only one of several), I will not say that this would speak for "all" or "many". We're a small shop and are not omniscient. In fact, I believe that MANY people would love native JSON support -- but not for the sake of being "happy", but instead for the sake of having a _measurable_ benefit as _our_ users have to pay for changes, and will not pay if it does not provide much, as they _all_ use Java EE hence have JSONP API RI on their classpath already. Also I'm an active JCP member so certainly I am focused on JDBC more than on proprietary features. I still do not see ANY difficulties by adding one more dependency to an application and what you propose also does not provide that "great" user experience in the global view of an application with all its lots of dependencies and complexity, but this is just my own opinion and as David already said, he anyways wants to provide "something working" anyway, so why don't we simply stop here with this fruitless repeating of the same arguments? All is said alaready, and we both do not add anymore benefit in this thread. It is for David to decide now how he likes to get things done. I'm pretty sure he understood you just as he understood me.

 

One last thing: You explain that you want to provide better user friendlyness as others. So can you please explain how that others (nemaly Oracle, MSSQL, Sybase, etc.) allow you to get a JSON-Object out of their JDBC drivers? I like to learn what is bad with their way to do it and why your proposal is better than theirs. Just to write on the box "supports JSON" is something I'd leave to the marketing guys, I'm a programmer. ;-) I really want to compete with MongoDB, but using real beneficial features, not simply using buzzwords ("me-too marketing"), and not for the sake of possibly getting in trouble with future JDBC 5 incomplience. :-)

 

Regards

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Montag, 29. Juni 2015 22:44
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 29/06/15 21:41, Markus KARG wrote:

You're overdramatizing things and I did not want to offend you (sorry if you feel that way, it is not on purpose). Actually I meant it literally when I said "just to make you happy",


    Markus, I don't want to keep on engaging in non-productive debates, but again I think you should moderate your language. Saying now I am overdramatizing is a realization of this.


as I do not see any other reason to provide a solution, as it does not bring ANY benefit to the user BUT implies future problems as I and others already explained.

    Maybe I am not explaining myself, but that you doesn't understand what I say doesn't mean you have to reduce my goals to the realm of myself, specially if I have very clearly and loudly stated that I want something for all users, or most users. So ignoring my references is not a nice summary of my speech.


I do not know on what basis you say you're speaking for "all" users (if you are a PostgreSQL official in some way or have done a poll please just tell me),


    It's an unofficial, belief-based approach. But follows a logical perspective, and is also based on true conversations with many users, with which I am in permanent contact. And everybody (mostly) wants to use JSON in a natural way from the drivers.


and I do not see what harm it does to the good work done INSIDE of PostgreSQL wrt JSON if pgjdbc does not provide JsonValue, as I explained (two times meanwhile) already that you can use that great work with the EXISTING driver.


    Not without difficulties. Not without a great user-experience.


Also I do not see in what way a user is forced to use MongoDB just because we like to wait with a driver change until the JDBC working group decided to add JSON. I do not see how you "help" users just by moving the parser from the application into the driver, actually.

    You care a lot about performance. That's great. But unfortunately, user-friendliness, and featureset, are mostly always preferred by users over extreme performance. And very fast products that nobody use end up going away.

    All I want is to compete, with a better product, but with at least the same user-friendliness as other, possibly competing alternatives in this area. And this requires the driver to "support JSON", whatever that means (I think what it means, but I may be wrong).


 

Sorry I totally do not see your complete point in all what your write. Maybe something else can chime in and explaint so I see Alvaro's point?


    I think Dave did a way better job explaining what I mean.

    Having that said, I insist that you will have me by your side to improve JSON at the protocol level and include any optimizations there. Just ping me back by then.

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

Thanks

Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Montag, 29. Juni 2015 21:02
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.



I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....



 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.


    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro





-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro






-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 







 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.





  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.






  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro







-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 









--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)






 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)






 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)





 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp






 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro






-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 

Re: SQLJSON

From
"Markus KARG"
Date:

Dave,

 

I hear you, but I simply do not see who tries to break something here. We just provide information to you, and then it's up to you to decide. :-)

 

Regards

-Markus

 

From: davecramer@gmail.com [mailto:davecramer@gmail.com] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 22:59
To: Markus KARG
Cc: List
Subject: Re: [JDBC] SQLJSON

 

On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?

 

Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.  

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)

 

Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)

 

As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.

 

Regards,

 

Dave 

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.

I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....

 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro

-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 



 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.

  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.


  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 





--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)


 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)


 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)

 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp


 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 

 

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 29/06/15 23:39, Markus KARG wrote:

Álvaro, please keep calm,



    Markus:

    You first said I was proposing only to satisfy myself; then you said that I was overdramatizing. Now you say I should keep calm.

    All these are value judgments, and they are not respectful with each other opinion's and the nettiquete for a responsible debate on an online forum.

    Your knowledge and contributions are very welcome. This attitude is not.

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata

Re: SQLJSON

From
Vladimir Sitnikov
Date:
>For what it's worth, I think bundling the JSONP API with the driver is a bad idea

+1

>To be clear I'm totally fine with a Maven dependency on whatever JSR spec jar is needed,

+1.  Probably provided/optional maven scopes could help to isolate
driver's part that implements "json support".

Regarding "getObject", the expected java.lang.Class is provided either
via "Connection.getTypeMap" or explicitly (like in
java.sql.ResultSet#getObject(int, java.lang.Class<T>)).
In other words:
1) If current driver returns String and "after json improvement" it
would start to return JsonValue, then it is likely a
backward-incompatible change. If we go for it, we would probably need
a setting to enable backward-compatible mode (e.g.
"return_string_for_json")
2) getObject(index, JsonValue.class) is somewhat reasonable API to
fetch JsonValue out of a ResultSet. I think it is fine if that
particular call would succeed only if "pgjdbc-json-support.jar" is
added to the list of dependencies (e.g. classpath).

>I just don't want that crap in my Maven build,

+1

>I would not be opposed to having "Driver rollup" bundles like a "jdbc + jsonp-spec + jsonp-ri" jar.

I would :)

Vladimir


Re: SQLJSON

From
Vladimir Sitnikov
Date:
Álvaro, Dave,

If I understand you right, you "just" want to make "usage of json
feature" easy for end-users.
So do I.
If there is more, please add.

What if we do the following?
1) We keep "base" part of driver "unaware" of json. In other words,
calls like getObject(1, JsonValue.class) would end up in "@throws
SQLException if conversion is not supported" (as per ResultSet's
javadoc)
2) Add "pgjdbc-json" module (i.e. jar) that adds support for
getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
3) Document "best choice of json dependencies" right in the readme. I
think we would be fine even with simple "we tested just jackson and it
works with pgjdbc".

From the end-user perspective it will be:
1) Using driver as usual -- "just add one mvn dependency" --
'org.postgresql:postgresql:9.4-1201-jdbc41'
2) Adding json support -- add additional one --
'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.

Both items would be available on the top of readme as copy&paste ready snippets.

This covers "json support" and it is user-friendly: no additional
googling is required to use the feature.
Am I missing anything?

Vladimir


Re: SQLJSON

From
Dave Cramer
Date:

On 29 June 2015 at 19:00, Vladimir Sitnikov <sitnikov.vladimir@gmail.com> wrote:
Álvaro, Dave,

If I understand you right, you "just" want to make "usage of json
feature" easy for end-users.
So do I.
If there is more, please add.

What if we do the following?
1) We keep "base" part of driver "unaware" of json. In other words,
calls like getObject(1, JsonValue.class) would end up in "@throws
SQLException if conversion is not supported" (as per ResultSet's
javadoc)
2) Add "pgjdbc-json" module (i.e. jar) that adds support for
getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
3) Document "best choice of json dependencies" right in the readme. I
think we would be fine even with simple "we tested just jackson and it
works with pgjdbc".

From the end-user perspective it will be:
1) Using driver as usual -- "just add one mvn dependency" --
'org.postgresql:postgresql:9.4-1201-jdbc41'
2) Adding json support -- add additional one --
'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.

Both items would be available on the top of readme as copy&paste ready snippets.

adding support with maven is the least problematic solution; Not everyone uses maven unfortunately.
It would appear we will need more than one jar  

This covers "json support" and it is user-friendly: no additional
googling is required to use the feature.
Am I missing anything?

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
     Hi Vladimir.

On 30/06/15 01:00, Vladimir Sitnikov wrote:
> Álvaro, Dave,
>
> If I understand you right, you "just" want to make "usage of json
> feature" easy for end-users.
> So do I.
> If there is more, please add.
     IMHO, it's nothing more than that :)

>
> What if we do the following?
> 1) We keep "base" part of driver "unaware" of json. In other words,
> calls like getObject(1, JsonValue.class) would end up in "@throws
> SQLException if conversion is not supported" (as per ResultSet's
> javadoc)
     I am not against having a json-unaware version of the driver, but I
wouldn't make it the default. To me, a RuntimeException over an expected
behavior is not user-friendly. So this version would be either for users
that don't want JSON at all, or for very advanced users that want to
supply other potentially conflicting versions of JSR353.
> 2) Add "pgjdbc-json" module (i.e. jar) that adds support for
> getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
     I would make this the default, aka "the driver".

> 3) Document "best choice of json dependencies" right in the readme. I
> think we would be fine even with simple "we tested just jackson and it
> works with pgjdbc".
>
>  From the end-user perspective it will be:
> 1) Using driver as usual -- "just add one mvn dependency" --
> 'org.postgresql:postgresql:9.4-1201-jdbc41'
> 2) Adding json support -- add additional one --
> 'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
> 'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.
>
> Both items would be available on the top of readme as copy&paste ready snippets.
>
> This covers "json support" and it is user-friendly: no additional
> googling is required to use the feature.
> Am I missing anything?
>
> Vladimir

     As I mentioned, my opinion is that the default is the one that
works without throwing an exception. You plug it in, and it works. This
is what I believe users would expect.

     As there's debate, I think we should approach it from an analysis
perspective: let's build a list of pros and cons and then compare.
Because all the arguments against bundling the API are basically avoid
potential conflicts, but in my opinion that is extremely unlikely and
hence not practical to address. Any future JSR353 spec would probably
require many changes anyway (if JSR353 ever changes, it would probably
be to be integrated in Java10). And bundling the RI is just adding 64Kb,
and wrapping the code in different classes as Dave suggested, to avoid
any conflicts.

     So the JDBC+API+wrapped RI is a bundle that just works, weights
just 96Kb more than the JDBC current driver and would only cause
conflicts if JSR353 is ever updated. But it provides a hassle free, no
googling required, works out-of-the-box JSON experience. Then we could
also package as non-default, specialized versions, one without the RI
and one without API and without the RI.

     If there are further inconveniences in making the API+RI bundle the
default, please specify.

     Thanks,

     Álvaro

--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
Vitalii Tymchyshyn
Date:

As for me, the only problem is with getObject(i) call and I'd really prefer it to be backwards compatible (return String) unless some parameter is set.
As of getObject(i, JsonValue.class) I don't see any problem implementing it as a single jar. And if user calls you with this class, he obviously does have at least the API jar in his classpath. Its fairy easy to make an implementation that wont fail in both cases (no JsonValue in classpath - no support, has JsonValue - support is enabled when demanded with the respectitive getObject call).

Best regards, Vitalii Tymchyshyn

Пн, 29 черв. 2015 19:34 Dave Cramer <pg@fastcrypt.com> пише:
On 29 June 2015 at 19:00, Vladimir Sitnikov <sitnikov.vladimir@gmail.com> wrote:
Álvaro, Dave,

If I understand you right, you "just" want to make "usage of json
feature" easy for end-users.
So do I.
If there is more, please add.

What if we do the following?
1) We keep "base" part of driver "unaware" of json. In other words,
calls like getObject(1, JsonValue.class) would end up in "@throws
SQLException if conversion is not supported" (as per ResultSet's
javadoc)
2) Add "pgjdbc-json" module (i.e. jar) that adds support for
getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
3) Document "best choice of json dependencies" right in the readme. I
think we would be fine even with simple "we tested just jackson and it
works with pgjdbc".

From the end-user perspective it will be:
1) Using driver as usual -- "just add one mvn dependency" --
'org.postgresql:postgresql:9.4-1201-jdbc41'
2) Adding json support -- add additional one --
'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.

Both items would be available on the top of readme as copy&paste ready snippets.

adding support with maven is the least problematic solution; Not everyone uses maven unfortunately.
It would appear we will need more than one jar  

This covers "json support" and it is user-friendly: no additional
googling is required to use the feature.
Am I missing anything?

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Christopher BROWN
Date:
Given the proposal, I think it would be better to simply provide a "how to" document showing how to take that output and feed it into the JSONP API, with the RI.

I can't see any advantage in adding maintenance overhead (for example, if the JSONP API/RI next release requires say Java 8/9, and the policy is that it's bundled, you're going to have to make this the minimum for PostgreSQL).  For standalone applications, you might have one less dependency, but you're going to risk classloader conflicts in enterprise applications and hard-to-diagnose support requests.  In other words, it'll help some users, but hinder others.

There's a strong risk of conflict with future JDBC APIs, which will undoubtedly add JSONP support at some point, given that MySQL and SQL Server seem to be going down this road.  That's another (avoidable) maintenance overhead.

And as has already been pointed out by others, using extra methods on specific PGxxx classes isn't practical in conjunction with connection pools and their associated wrappers, even if it's less risky than trying to anticipate what might be the best way to implement getObject().

The only real reason, other than "we must do something, anything", would be either (a) JDBC mandates it or (b) the protocol provides an efficient way of streaming data without buffering it (for example, streaming JSON directly from the database, via an application server, to a browser).  In which case, having the **choice** of a "JSON DOM" or a streaming parser / generator would still be important, not just one or the other based on guesses about what may or may not be useful.

I understand the "just works" point of view, along with "we must do something, anything", but in a real-world application deployed to one of our customers, there's an API that we currently can't work around that provides easy access to an XML DOM from an arbitrary database.  And it's causing us a lot of pain.  Buffering real-world data (and it's just an enterprise application, not "web scale") is causing lots of throughput issues (how many requests we can handle, not just raw performance), garbage collector thrashing, and humongous object allocations (yes, that's actually a term, and it's what you get when large datasets are buffered in memory).

I think a lot of valid concerns have been raised, and I can't understand the rush to do anything that might be a problem later.  And all this for something that can be explained in a short example, even to hipsters.  Other databases have this approach, and I can't foresee an exodus of developers leaving all ACID-compliant databases for MongoDB (and following the learning curve) just because they can't read / find a few lines (with links) on how to read a string with a JSON parser.  If they know the API already, I can't see how they can't apply it to a string from a database. 

--
Christopher


On 29 June 2015 at 23:21, Dave Cramer <pg@fastcrypt.com> wrote:



On 29 June 2015 at 17:19, Christopher BROWN <brown@reflexe.fr> wrote:
Hello,

In summary, is this what is planned ?
  • Bundle the JSONP API with the driver (specific version)
  • Bundle the JSONP RI with the driver (specific version, with modified package names)
  • Override getObject() to return a "JSON DOM" (buffering data once, then reconstructing it again as an object model)
Is that correct?  Could you remind me what does getObject() currently return for JSONB (I don't have a test case to hand)?

That is the current proposal, yes. I would expect getObject to return a string at this point. But I haven't tested it either

Dave 

Thanks,
Christopher


On 29 June 2015 at 22:58, Dave Cramer <pg@fastcrypt.com> wrote:
On 29 June 2015 at 16:22, Markus KARG <markus@headcrashing.eu> wrote:

Dave,

 

I understand that people expect that getObject does something reasonable - but the case why this does not happen is because PostgreSQL does not provide something reasonable AFAIK, or is there JSON support in the protocol (I might have missed that, actually, then please just tell me)?


Well things in PostgreSQL  move at rather a glacial pace as far as features go. The emphasis is on reliability. If you think this mailing list is contentious you might want to have a look at hackers. To answer your question though there is no special backend support for JSON, or any other streaming.  Yes this is a performance drawback, but it is what it is.  

 

But I need to tell you that you have to deal with the risk that if JDBC 5 comes up with the definition "getObject has to return FancyJsonType" you'll face a backwards compatibility problem when you decided for pretty anything to return but not "FancyJsonType", whether the type you've choosen looked obvious to the common user, to almost any user or to nobody at all. Anyways, you're the boss, and I trust in you to do the right choice. :-)


Doing nothing is surely the wrong choice. The question is what to do. I don't pretend to know the right answer here.However in the absence of the spec, I'm sure whatever we do will be a compromise.

 

Regarding your metaphor, please in turn understand thar some guys in this form, like Vladimir and me, are professional combustion engine engineers in our main job. Our whole life is dedicated to the difference between a vanilla BMW 318 into an M3. I hope there is place for such engineers in your team, too, and it is _not_really_ your attitude that you are happy once the weels don't fall away once you start the motor. We do not do it for thank or cheer, we simply do it because it is the attitude of our main jobs. We cannot forget our knowledge about compression and consumption just because 99% of the users don't care for. And part of this knowledge is thinking of efficiency and safety and cost of production and unassembly for recycling, and usefulness and maintainability and testability of the car, too, even if no user _ever_ would think about when sitting is his ride. If our demands on the professionality and performance of the end product is not what you like to achieve and you like to simply replace the pipe with a chromed one, then please let us know. Otherwise our PRs will one day produce a stack overflow on GitHub, which nobody wants. ;-)


As I said in my earlier email I am grateful for your contributions and I would love them to continue, but we need to keep the majority of the users in mind. We also need to make sure in our quest for performance that we don't break anything.

Regards,

Dave 

 

Have fun

-Markus

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Montag, 29. Juni 2015 21:23
To: Álvaro Hernández Tortosa
Cc: List
Subject: Re: [JDBC] SQLJSON

 

Markus,

 

I really value your recent input to the driver and would like it to continue but we need to keep things in perspective. The mere fact that you are engaging in discussion on this list puts you in the 99.99999th (keep going with 9's if you like) percentile of users. Most and I mean the rest of the people in that population just expect the driver to do something reasonable. They don't provide JSON parsers or even know there is a difference. When they do getObject and that object is a json value they expect something magical to happen. We return an object they can use.

 

The other less than exciting fact about the driver is that it is not the central feature of PostgreSQL. Allow me a metaphor if you will. I like cars so I'll chose to use a driveshaft. It connects the engine to the wheels. There is nothing new about driveshafts, although recently they have been made out of carbon fibre, but essentially they are tubes which connect the crankshaft to the wheels. To continue the metaphor, when the driver applies the gas pedal they expect the car to move. If the driveshaft breaks then this doesn't happen. Our job is to make sure the driveshaft continues to connect the engine to the wheels. If we can make it out of carbon fibre so much the better, but it is a particularly thankless job.

 

 

Kind Regards,


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 29 June 2015 at 15:01, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:


    Markus:

On 29/06/15 18:26, Markus KARG wrote:

Unfortunately, yes, it is weird to ask, as what you do actually is working already once you accept the technical reality that an application either has to use JSON SQL functions OR provide an implementation of the JSONP API to be able to process JsonObject.


    It was rhetorical. Precisely I'm saying let's include an implementation so that you could process a JsonValue. Of course.


I actually cannot see what is so specific to PostgreSQL that pgjdbc has to support a use case that no other driver supports.


    Maybe that PostgreSQL has JSON, really good one, while others don't? Or let's ruin all the work done into JSON just because you fear of an absolutely unrealistic class path? Really? Please....


 

The argument against it is that it provides classpath clashes once pgjdbc is used in a Java EE environment, and that our solution might be incompatible with JDBC 5, and that our solution imposes work and complexity on the pgjdbc contributors just for the sake to make you happy, without providing you any measurable benefit.


    Markus, when you say "to make you happy"....  please take back your words and stick to rational arguments. If you cannot, please abandon this otherwise respectful and enriching debate.

    I'm speaking for all the users, all of them that want to use 9.4's best thing (with logical decoding permission, of course).

    Yet you only want to stop everything speaking of a class path that very likely no one will ever experience, or some fears about JDBC5 which might be light years ahead. We will worry about JDBC5 then, but now we need to help our users, not help them (with your help) to go to MongoDB.




    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 21:06
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 17:09, Markus KARG wrote:

If your application does not want to deal with JSON data then don't request it from the database, but use http://www.postgresql.org/docs/9.4/static/functions-json.html in your SELECT statement. Performing a JSON extraction function natively on the server and simply use the existing data types in your application not only reliefs us from dealing with JSON in the driver, but also will work magnitudes faster particularly on congested networks. That does work by default already in the existing driver. What you ask for does not provide any benefit to the user from my view, or I still do not understand you exact scenario.


    Markus, here you are trying to tell users what to do. I prefer to give them freedom, works best.

    All I'm saying is:

- Give users an API to get a javax.json.JsonValue out of a PostgreSQL column which is of type json, jsonb or the result of an expression which evaluates to any of those.

- Embed a JSON parser in the driver (like the RI implementation, which adds only 64Kb) so that users don't need to load any other code, unless they want to override the default JSON parser.

    From there, I don't care whether the user uses JSON functions at the server or the JSON API. What I want is that it works by default and that you can query JSON. Is this that weird to ask?

    I'm sorry, I fail to understand your solution to the problem. Would you please mind elaborating on the drawbacks of my proposal? I stated in previous emails the drawbacks I see on not doing it, but I still fail to see an argument against this.

    Thanks,

    Álvaro


-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Sonntag, 28. Juni 2015 11:57
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 11:49, Markus KARG wrote:

You miss one essential point against bundling: An application author who wants to process JsonObject must have that class on his classpath, even if he does not use JDBC at all. This essential is HIS problem as HE wants to process it. So where does he get that class from other than putting any JSR 253 implementation in his classpath? Again, it is simply not pgjdbc's problem as in that scenario no JDBC is used at all.

    I don't agree, it's not a must. I may want to extract JSON data from the database and then manipulate it as non-JSON data. It all depends on your domain Objects.

    I already stated how bad for the user is not to have a driver that works by default. It may be as hard as adding a 64Kb to the driver. I don't understand how is this a problem, vs. the problem it creates for the user.

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 




 

From: Álvaro Hernández Tortosa [mailto:aht@8Kdata.com]
Sent: Sonntag, 28. Juni 2015 11:41
To: Christopher BROWN
Cc: Sehrope Sarkuni; Markus KARG; Dave Cramer; List
Subject: Re: [JDBC] SQLJSON

 

 

On 28/06/15 09:34, Christopher BROWN wrote:

Hello,

 

Quick contribution, I'm not answering in-line because there are already too many in-line answers and it's getting unreadable.

  • In my own applications, I use Jackson, but it's many up of different ".jar" files and has therefore no definitive form (you can concoct lots of combinations).  It's also quite heavy in terms of footprint, and embedding it makes no sense, because you'd have to keep updating the driver to keep up to date with Jackson. Finally, it doesn't actually implement JSR353 (although it would be possible to create a thin wrapper), out-of-the-box (via a compatibility API) it can read JSR-353 but it basically rebuilds a Jackson representation out of a "standard" representation.  I might choose Jackson, but I wouldn't want to impose it or require that it be bundled with the driver (indeed, that would cause me classloader issues as I often update to the latest version of Jackson).

    Although I mentioned Jackson as a good candidate for a default implementation, you are right it does not implement JSR353 directly (although wrappers already exist, like https://github.com/pgelinas/jackson-javax-json). But it does not need to be the default choice. I did a quick test and wrapping Jackson with jaackson-javax-json and the set of dependencies to make it work would add 1102Kb to the Jar file. Not much IMHO, although bigger than current driver size. I would not be scared however to see a 2Mb jdbc jar file.

    However, the Reference Implementation (https://jsonp.java.net/) is probably good enough and only adds 64Kb to the Jar file. The JSR353 is just another 32Kb, so in total 96Kb would be added if using the RI rather than Jackson. I don't consider this offensive.

    In summary: why not bundle then the RI? Works out of the box and does not conflict with Jackson. Want to use Jackson? Go for it. The rest of the world would have something running out-of-the-box.


  • You can compile the driver against the JSONP API without embedding either the interfaces or an implementation.  It's therefore an optional feature for those that require it, and it's not rocket science to add the necessary APIs to the classpath.
  • I disagree that bundling interfaces + implementation is "making it easy".  For some users, perhaps, but for others, you're going to cause headaches due to creating classloader conflicts (when it's already bundled in their application).

    Technically, it's possible. But from a user perspective, not bundling an implementation means:

- Understanding why the code I wrote fails with a "Provider org.glassfish.json.JsonProviderImpl not found".
- Google that. Find that you need to add the dependency and very likely create a SPI file (META-INF/services/javax.json.spi.JsonProvider).
- Blame PostgreSQL for not doing that by default and shipping a half-baked driver that is not finished (won't be my opinion, but might be user's opinion).
- Google again to see what JSR353-compliant implementations are out there.
- Blame PostgreSQL again for not making this choice for you.
- Struggling to find a compatible implementation. Find Jackson but realize requires third-party driver. Question the "quality" of that wrapper and consider whether that would be "supported" with PostgreSQL driver.
- Luckily enough a search points you to a StackOverflow link that suggests to use either this Jackson wrapper or the Reference Implementation (there are not many JSR353 implementations, after all).
- Choose without knowing which JSON parser is best.
- Bundle the dependency, check now it works. Clean your sweat.
- Wonder why not to choose MongoDB next time, it works out of the box.

    Not funny.

    The alternative is: bundle a default (the RI) and let advanced users not happy with the default implementation to make another choice (basically create the META-INF/services/javax.json.spi.JsonProvider and that will override the bundled RI). You have the best of both worlds.

    Pareto (https://en.wikipedia.org/wiki/Pareto_principle): let's make it work for 80% and let that 20% to edit a file and "suffer" a 64Kb size increase in the driver, rather than ask everybody to go through the above process.



  • If as Dave Cramer says, the PG protocol doesn't currently support streaming, it still makes sense to add streaming support that reads from already fully-read resources... because in that way, if the protocol improves in the future, client code using the streaming API will benefit (with no changes to client code) in the future.

    JSR353 already has a Streaming API. I don't understand what do we need to do here, it's already done.

    Best regards,

    Álvaro




-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 






--

Christopher

 

 

On 28 June 2015 at 01:53, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

 

On 28/06/15 00:55, Sehrope Sarkuni wrote:

On Sat, Jun 27, 2015 at 6:25 PM, Álvaro Hernández Tortosa <aht@8kdata.com> wrote:

    Hi Sehrope!

 

Hi Álvaro! :D

 

    To me, this is the least important question. If based on JSR353's SPI, it's trivial to swap the default, included one, for another one. Just picking a sensible default (Jackson, for instance) is probably good enough.

 

I think I've used Jackson almost every time I've had to deal with JSON in Java. The mapping API is pretty cool in that it lets you directly create an target object type. If we got the route of adding methods to PGResultSet then we could have something like: <T> T getJsonAsType(String, Class<T> clazz)


    That might be a nice addition. But I believe that goes beyond driver's responsibility: I think it ends when it returns you the JSON type you queried (JsonObject in my previous email, but I'm correcting now myself: JsonValue)



 

I'm not wedded to Jackson though. Honestly if JS353 is the standard then that's what we should be using. We'd still need to figure out how to handle older JVMs or maybe just selectively disable the feature (JDK8+?).


    JSR353 is targeted for JavaSE 6 :)



 


#2 is driven a lot by #1 as depending on the parser implementation there may be different object types returned. JSON is a bit tricky as "valid JSON" can mean null, a scalar, an object, or an array. Most people thing of it as just an object but "foobar" is valid JSON as well. This leads us to #3...


    The object type to return has to be IMHO JsonObject: http://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html

 

Not always though. All these are valid JSON too: 

 

=> SELECT '1'::json AS num, '"test"'::json AS string, '[1,2,3]'::json AS arr, '{"foo":"bar"}'::json AS obj;

 num | string |   arr   |      obj      

-----+--------+---------+---------------

 1   | "test" | [1,2,3] | {"foo":"bar"}

(1 row)

 

We'll need separate getters/setters for the scalar and array types as well. I agree that most people will just be using the object type though (and maybe the array).


    You are right here. Please s/JsonObject/JsonValue/g JsonValue is a container for any of the above including objects and arrays. So it would be enough just with JsonValue getJsonValue(....)


 


#3 doesn't have a straight answer as there is no getJSON(...) methods in the JDBC spec. It'd probably have to be returned via getObject(...).

An alternative is to provide PGResultSet and PGPreparedStatement classes similar to PGConnection that provides PG extensions. They could have the get/set methods (ex: getJsonScalar(...) or setJsonObject(Map<String,Object> ...)) to retrieve JSON values as specific object types (i.e. scalar, object, array). It'd be a bit more type safe as presumably most people using json/jsonb types know the top level type of what they're storing.


    Probably adding methods to PG classes would be better than getObject and force explicit casts. Regarding the methods, if they simply return JsonObject, you already have a full API there to parse and extract and process. So anything that returns a JsonObject from a column (identifier or #) would be enough for me.

 

For most cases I think it'd be fine. I think the custom mapping I mentioned above would cover the rest. Anything beyond that would be a full on transformation and would be very application specific.


    Yepp



 

For #4 I see two possible wins. First off on the usability side, there's some convenience to natively interfacing with json/jsonb types. It'll only have value though if those types are the same ones that users are using in the rest of their code. If they're just using them as Map<String,Object> everywhere then it'd still be a pain for a user to convert to our "native" PG JSON types to use via JDBC. Having a dedicated API that allows for interaction using native Java types would make this more convenient.

The other win I can see for #4 is on performance. Right now JSON is converted to a String. That means everybody using it has to convert it twice. First raw bytes to String, then String to object. A dedicated API could cut one of those out of the way. Given how the wire protocol is implemented in the driver, it wouldn't be a direct reading from the input stream (it'll be buffered in a byte array), but at least it won't be copied twice.


    As far as I know, most users are using JsonObject, so returning that is a perfect match for pgjdbc. I don't expect however big performance wins as JSON is sent as a String over the wire...

 

The performance gain isn't on the wire, it's from not having to convert bytes => String => JsonObject. It'd be bytes => JsonObject or bytes => CustomObject. Less work and less GC. The bigger the JSON string, the bigger the savings too.


    You are right in that JSR353 allows you to create a parser directly out of an InputStream, so you would avoid converting to String. That's a win. The rest of the conversions are inevitable (having the latter one you pointed out laying in user's realm, beyond driver's responsibility).

    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

 

 

 

 

 





Re: SQLJSON

From
Vladimir Sitnikov
Date:
> So the JDBC+API+wrapped RI is a bundle that just works

I wish some OSGi expect chimed in and told us if such bundling is
workable in OSGi world.

>  If there are further inconveniences in making the API+RI bundle the default, please specify.

1) I just do not see why you feel "using several dependencies" to be a
lot harder than "using all-in-one-bundle".
I agree it is a bit more work, but it is not a rocket science.

If user codes against JsonValue, then she expects to have that API in
the class path. I would say "jdbc driver" as a provider of "JSR353"
would violate the principle of least astonishment.

2) For instance: slf4j, log4j, logback do not have default bundles.

3) If each and every library bundles JSON API, we would get
LinkageErrors or similar errors soon. It is just bad when distinct jar
files contain "the same" class files. It is very painful when they end
up to contain _different_ class files.

4) Consider the other side of a coin.
What if we would want implementing IBM's JSONx later?
(https://twitter.com/danharper7/status/514822464673951744).

If we had some way of "adding custom getObject" via external
dependencies, that would be much easier since we would not have to
touch "core" part of the driver.
If we hard-code JSR353 into the core "just for convenience of a single
jar", then it won't help us adding new datatypes.


Vladimir


Re: SQLJSON

From
Stephen Nelson
Date:

+1

With the caveat that this is the least worst option. Json is just text so it can be easily parsed by your favourite json library.

If I'm writing a web api which exposes json services I would be annoyed if my jdbc driver has decided which json library I'll use for my web api, or I'd have to use two different libraries and bloat my jar/war.

On Tue, 30 Jun 2015 00:01 Vladimir Sitnikov <sitnikov.vladimir@gmail.com> wrote:
Álvaro, Dave,

If I understand you right, you "just" want to make "usage of json
feature" easy for end-users.
So do I.
If there is more, please add.

What if we do the following?
1) We keep "base" part of driver "unaware" of json. In other words,
calls like getObject(1, JsonValue.class) would end up in "@throws
SQLException if conversion is not supported" (as per ResultSet's
javadoc)
2) Add "pgjdbc-json" module (i.e. jar) that adds support for
getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
3) Document "best choice of json dependencies" right in the readme. I
think we would be fine even with simple "we tested just jackson and it
works with pgjdbc".

From the end-user perspective it will be:
1) Using driver as usual -- "just add one mvn dependency" --
'org.postgresql:postgresql:9.4-1201-jdbc41'
2) Adding json support -- add additional one --
'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.

Both items would be available on the top of readme as copy&paste ready snippets.

This covers "json support" and it is user-friendly: no additional
googling is required to use the feature.
Am I missing anything?

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

On 30/06/15 09:07, Stephen Nelson wrote:

+1

With the caveat that this is the least worst option. Json is just text so it can be easily parsed by your favourite json library.


    Stephen, the idea is to provide "built-in" support for Json out of the database, so that you don't need to parse it manually, which is a pain for many users. Plus, it opens the door for future optimizations is Json support is streamlined into the protocol.

If I'm writing a web api which exposes json services I would be annoyed if my jdbc driver has decided which json library I'll use for my web api, or I'd have to use two different libraries and bloat my jar/war.


    The current proposal does not force you, at all, to decide which JSON library to use. It returns a JsonValue, which is a JSR spec, and then you can do whatever you want. The RI is just 64Kb, so I wouldn't call it bloat.... and in exchange for that, you get a fully working driver with JSON support from the beginning.

    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata



On Tue, 30 Jun 2015 00:01 Vladimir Sitnikov <sitnikov.vladimir@gmail.com> wrote:
Álvaro, Dave,

If I understand you right, you "just" want to make "usage of json
feature" easy for end-users.
So do I.
If there is more, please add.

What if we do the following?
1) We keep "base" part of driver "unaware" of json. In other words,
calls like getObject(1, JsonValue.class) would end up in "@throws
SQLException if conversion is not supported" (as per ResultSet's
javadoc)
2) Add "pgjdbc-json" module (i.e. jar) that adds support for
getObject(1, JsonValue.class), setObject(1, JsonValue) kind of calls.
3) Document "best choice of json dependencies" right in the readme. I
think we would be fine even with simple "we tested just jackson and it
works with pgjdbc".

From the end-user perspective it will be:
1) Using driver as usual -- "just add one mvn dependency" --
'org.postgresql:postgresql:9.4-1201-jdbc41'
2) Adding json support -- add additional one --
'org.postgresql:postgresql-json:9.4-1201-jdbc41' and
'best-of-the-best-pgjdbc-approved-json-impl:3.14.15'.

Both items would be available on the top of readme as copy&paste ready snippets.

This covers "json support" and it is user-friendly: no additional
googling is required to use the feature.
Am I missing anything?

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Vladimir Sitnikov
Date:
>  so that you don't need to parse it manually, which is a pain for many users

A) Can you please list pain points in literal?
I think we've already agreed that "having readme" is good enough so
even newbies can add dependencies.
What are the other pain points except "need to add a couple of jars if
json is needed"?

Frankly speaking, I find current "different jdbc version jars" even
more confusing than "if you want json, add pgjdbc-json dependency".

> so that you don't need to parse it manually,

B) That does not work that way.
Can you suggest a piece of "user code" you have in mind?

Let me predict it a bit:
1) It processes resultset
2) Fetches "JsonValue"
3) Passes to a business-logic class

We can stop here.
"business logic class" from #3 will definitely have "JsonValue"
classes in its method signatures.
In other words, it would require to have "javax.json" API in the class
path to compile.

Then I would love to know how is the user addressing that without
adding "javax.json" API dependency to her build system of choice.

The same business logic should be unit-tested (without DB), shouldn't it?
That results in "it should be testable without pgdjbc dependency".

C) If we continue creating bundles here and there (not sure what is
"non offensive" wording here), the number of possible bundles would
explode exponentially. jdbc driver is not an application for end-user,
so I do not see much reason why it should "bundle optional
dependencies without shading them".

For instance, if we decide to use objectweb.asm bytecode transformer,
it would be perfectly fine to shade it into org.postgresql.asm...
package, so it is not exposed to public.

However, with javax.json you cannot do package renaming since it would
make no sense.

D) Theoretically, we could represent json as java.util.Map /
java.util.List by embedding and repackaging some json implementation,
however I am not sure if that makes much sense provided JSR353 defines
proper interfaces.

Vladimir


Re: SQLJSON

From
Dave Cramer
Date:

On 30 June 2015 at 03:57, Vladimir Sitnikov <sitnikov.vladimir@gmail.com> wrote:
>  so that you don't need to parse it manually, which is a pain for many users

A) Can you please list pain points in literal?
I think we've already agreed that "having readme" is good enough so
even newbies can add dependencies.
What are the other pain points except "need to add a couple of jars if
json is needed"?

Frankly speaking, I find current "different jdbc version jars" even
more confusing than "if you want json, add pgjdbc-json dependency".


Yes, the multiple jars has caused confusion most notably in spring-core of all places 

> so that you don't need to parse it manually,

B) That does not work that way.
Can you suggest a piece of "user code" you have in mind?

Let me predict it a bit:
1) It processes resultset
2) Fetches "JsonValue"
3) Passes to a business-logic class

We can stop here.
"business logic class" from #3 will definitely have "JsonValue"
classes in its method signatures.
In other words, it would require to have "javax.json" API in the class
path to compile.

Then I would love to know how is the user addressing that without
adding "javax.json" API dependency to her build system of choice.

The same business logic should be unit-tested (without DB), shouldn't it?
That results in "it should be testable without pgdjbc dependency".

This is a valid point 

C) If we continue creating bundles here and there (not sure what is
"non offensive" wording here), the number of possible bundles would
explode exponentially. jdbc driver is not an application for end-user,
so I do not see much reason why it should "bundle optional
dependencies without shading them".

For instance, if we decide to use objectweb.asm bytecode transformer,
it would be perfectly fine to shade it into org.postgresql.asm...
package, so it is not exposed to public.

However, with javax.json you cannot do package renaming since it would
make no sense.

D) Theoretically, we could represent json as java.util.Map /
java.util.List by embedding and repackaging some json implementation,
however I am not sure if that makes much sense provided JSR353 defines
proper interfaces

This has little value IMO as the user is clearly using JSON and expects JSON.
returning a Map would just make matters worse


Someone noted that other drivers are returning JsonValue. Are they using a vendor specific class?

 
.

Vladimir


--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
Vladimir Sitnikov
Date:
>you need the API on your classpath. But that's obvious :)

Do you know of https://en.wikipedia.org/wiki/Principle_of_least_astonishment?
I would call an "astonishment" if my jdbc driver provided "javax.xml"
or "javax.json" or "java.util.concurrent" kind of packages.
Don't you think so?

>users don't realize they need another dependency.

This is trivially resolved via readme. Come on.
If a developer can't figure out he needs "a library for json", why do
we care to support such a developer?

Consider the following section in _current_ readme:

>Currently Waffle-JNA and its dependencies are required for SSPI authentication support (only supported on a JVM
runningon Windows). >Unless you're on Windows and using SSPI you can leave them out when you install the driver. 

So, there already exists a case when optional dependency is
recommended in readme. This might be improved in terms of "ready for
use mvn dependency tags", however it does not follow "bundle
everything" approach.

>Classpath collision is extremely unlikely, I won't bet on it, so why not help the user even more?

http://wiki.osgi.org/wiki/Split_Packages
Google finds 124000 results for "osgi split package".

Is that enough to stop embedding non-renamed packages?

The problem is not specific to OSGi. OSGi just highlights it.

Well, I guess it somehow can be not that convincing, so I throw
another issue on the table:
https://issues.jboss.org/browse/AS7-3305 (~
http://www.nightprogrammer.org/development/hints/illegalaccesserror-packages-classloaders/
)

>we could have provided instead with 0 drawbacks

Could you please reconsider that?

>Classpath collision is extremely unlikely

If the collision happens, we will have no reasonable way of solving that.
It's better to prevent that collision from the start than develop on a
minefield.

> No, it doesn't need to be like that. That you extract a JSON from your database, it doesn't mean that you have to
havea JSON on your business logic. Not at all. 

Please provide an example of business logic that extracts json values
and does not get infected its methods' signatures with json classes.
I can't follow your justification while you provide zero examples.

I hope you do not call "POJO processResultSet(ResultSet rs)" kind of
method a "business logic".
It is not testable since it does multiple things: "retrieves jsons and
performs conversion".
At least it should be "JsonValue processRow(ResultSet rs)" + "POJO
convertJsonToPojo(JsonValue)".

>That's it. 2. That's not an exponential explosion.

What do you think about Waffle-JNA then?
Should it be included in "all in one" bundle?
Should pgjdbc have "core", "core+waffle", "core+json", "core+waffle+json" jars?
It is exactly what I call "exponential explosion".

Vladimir


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 30/06/15 14:03, Vladimir Sitnikov wrote:
>> you need the API on your classpath. But that's obvious :)
> Do you know of https://en.wikipedia.org/wiki/Principle_of_least_astonishment?
> I would call an "astonishment" if my jdbc driver provided "javax.xml"
> or "javax.json" or "java.util.concurrent" kind of packages.
> Don't you think so?

     Bad examples. Both javax.xml and j.u.c. are already in JRE. I would
expect javax.json to be too, but it's not. However, it's a compile
dependency and hence I don't see how it would violate this principle.

>
>> users don't realize they need another dependency.
> This is trivially resolved via readme. Come on.
> If a developer can't figure out he needs "a library for json", why do
> we care to support such a developer?

     We should care about all, not become elitists. There are many cases
where you don't run through the documentation, but expect that something
works out-of-the-box.

>
> Consider the following section in _current_ readme:
>
>> Currently Waffle-JNA and its dependencies are required for SSPI authentication support (only supported on a JVM
runningon Windows). >Unless you're on Windows and using SSPI you can leave them out when you install the driver. 
> So, there already exists a case when optional dependency is
> recommended in readme. This might be improved in terms of "ready for
> use mvn dependency tags", however it does not follow "bundle
> everything" approach.

     Good. I think that could possible be improved, too :)

>
>> Classpath collision is extremely unlikely, I won't bet on it, so why not help the user even more?
> http://wiki.osgi.org/wiki/Split_Packages
> Google finds 124000 results for "osgi split package".
>
> Is that enough to stop embedding non-renamed packages?
>
> The problem is not specific to OSGi. OSGi just highlights it.
>
> Well, I guess it somehow can be not that convincing, so I throw
> another issue on the table:
> https://issues.jboss.org/browse/AS7-3305 (~
> http://www.nightprogrammer.org/development/hints/illegalaccesserror-packages-classloaders/
> )

     I am well aware of the problem. But I am not speaking from a
general perspective, but rather on a very specific case: a collision
would only happen if JSR353 is ever changed. I believe if that happens
is to embed it into JRE, and that will then require changes to the
driver anyway. So if the API is not embedded, then we are preventing a
problem that will very likely never occur. So why bother? (and if it
did, it would be as simple as doing a new driver release without the
dependency..... not a big deal)

>
>> we could have provided instead with 0 drawbacks
> Could you please reconsider that?
     As I said, I think the drawback is a very unlikely event, easily
solvable. And it helps a lot everybody, IMHO. Your opinion is different,
well, I respect that.
>
>> Classpath collision is extremely unlikely
> If the collision happens, we will have no reasonable way of solving that.
> It's better to prevent that collision from the start than develop on a
> minefield.

     An earth-sized minefield with just one mine. I'd take the risk.

     Oh, and if we hit the mine, the way of solving that would be to
have a new driver version, now without the dependency, when your point
is proven rather than guessed. Then, I will approve that immediately.
But until then I'd make life easier for everybody. Otherwise, it's over
engineering.

>
>> No, it doesn't need to be like that. That you extract a JSON from your database, it doesn't mean that you have to
havea JSON on your business logic. Not at all. 
> Please provide an example of business logic that extracts json values
> and does not get infected its methods' signatures with json classes.
> I can't follow your justification while you provide zero examples.

     Trivial. Extract values from Json and store in non-json fields,
transform Json to Maps, you name it. Anything where your business logic
does not store Json* references.

>
> I hope you do not call "POJO processResultSet(ResultSet rs)" kind of
> method a "business logic".
> It is not testable since it does multiple things: "retrieves jsons and
> performs conversion".
> At least it should be "JsonValue processRow(ResultSet rs)" + "POJO
> convertJsonToPojo(JsonValue)".

     No, I don't.

>
>> That's it. 2. That's not an exponential explosion.
> What do you think about Waffle-JNA then?
> Should it be included in "all in one" bundle?
> Should pgjdbc have "core", "core+waffle", "core+json", "core+waffle+json" jars?
> It is exactly what I call "exponential explosion".

     Honestly, I'm speaking about Json now, which is a widespread
request. I'm not that sure about waffle, it should probably be
considered on a case-by-case basis. In any case, I'm not saying let's do
a "core" and then "core+json". I would rather say "core" and "core minus
json", since I see very few reasons not to use the first one.

     All that said: my opinion is clearly there, I will leave the
decision to the rest of the list and the team. Hope that my 2 cents helped.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
"Markus KARG"
Date:

Álvaro, again, you simply misunderstood my words. I never wanted to offend you. Actually I didn't but you just undertood it as an offense. Sorry again, but it is not my attitude which is the problem here, but your assumption of what my attitude is. Isn't this enough misunderstandings right now and we can come back to a discussion on the actual topic?

 

-Markus

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Álvaro Hernández Tortosa
Sent: Dienstag, 30. Juni 2015 00:37
To: pgsql-jdbc@postgresql.org
Subject: Re: [JDBC] SQLJSON

 

 

On 29/06/15 23:39, Markus KARG wrote:

Álvaro, please keep calm,



    Markus:

    You first said I was proposing only to satisfy myself; then you said that I was overdramatizing. Now you say I should keep calm.

    All these are value judgments, and they are not respectful with each other opinion's and the nettiquete for a responsible debate on an online forum.

    Your knowledge and contributions are very welcome. This attitude is not.

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

Re: SQLJSON

From
Vladimir Sitnikov
Date:
ah. I meant to double-check with Álvaro if he is suggesting compile
type dependency.

If he means that we in fact are discussing the same thing, so no
contradiction exists.

> However, regarding POLA you say "compile dependency" which means you
> suggest _not_ including javax.json into pgjdbc.jar
>
> Álvaro , Can you please tell us if "using compile type dependency for both
> javax.json and RI" suits you?
>

Vladimir


Re: SQLJSON

From
Vladimir Sitnikov
Date:
I think we have all the input from Álvaro.

It boils down to "including javax.json classes into pgdjbc is fine,
and if it creates a problem we can release a new version that removes
the dependency". This is how I read your "In brief: users don't
realize they need another dependency"

However, regarding POLA you say "compile dependency" which means you
suggest _not_ including javax.json into pgjdbc.jar

Can you please tell us if "using compile type dependency for both
javax.json and RI" suits you?

Vladimir


Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
     Thanks for asking for the double-check. No, indeed I'm still asking
to provide the class files for the API in the package. I feel that's the
right way, and I don't see it would create conflicts unless JSR353 would
create a new version, something which I believe extremely unlikely until
it merges with Java10 or JDBC5 comes out, point at which we would need
to change things anyway.

     However, I don't want to insist more or suck more dev bandwitch,
that's my opinion and it's been stated more times than I wish, so I
would now leave the decision to the rest of you :)

     Regards,

     Alvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



On 30/06/15 18:49, Vladimir Sitnikov wrote:
> ah. I meant to double-check with Álvaro if he is suggesting compile
> type dependency.
>
> If he means that we in fact are discussing the same thing, so no
> contradiction exists.
>
>> However, regarding POLA you say "compile dependency" which means you
>> suggest _not_ including javax.json into pgjdbc.jar
>>
>> Álvaro , Can you please tell us if "using compile type dependency for both
>> javax.json and RI" suits you?
>>
> Vladimir



Re: SQLJSON

From
Steven Schlansker
Date:
On Jun 30, 2015, at 10:25 AM, Álvaro Hernández Tortosa <aht@8Kdata.com> wrote:

>
>    Thanks for asking for the double-check. No, indeed I'm still asking to provide the class files for the API in the
package.I feel that's the right way, and I don't see it would create conflicts unless JSR353 would create a new
version,something which I believe extremely unlikely until it merges with Java10 or JDBC5 comes out, point at which we
wouldneed to change things anyway. 

I do not believe this is as unlikely as you think.  For example, the javax.ws.rs JAX-RS spec has 22 different versions
availableon Maven Central.  Granted, many of these are milestone builds and not releases, but there are already two
majorversions, a patch available and a new minor version coming through the pipes. 

So assuming these spec jars will not change is provably false.  One of my fears is that if PG bundles JSR353 1.0, and I
needJSR353 1.0.1, I now need to fight the driver packaging to upgrade a theoretically unrelated package. 

>
>    However, I don't want to insist more or suck more dev bandwitch, that's my opinion and it's been stated more times
thanI wish, so I would now leave the decision to the rest of you :) 
>
>    Regards,
>
>    Alvaro
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
> On 30/06/15 18:49, Vladimir Sitnikov wrote:
>> ah. I meant to double-check with Álvaro if he is suggesting compile
>> type dependency.
>>
>> If he means that we in fact are discussing the same thing, so no
>> contradiction exists.
>>
>>> However, regarding POLA you say "compile dependency" which means you
>>> suggest _not_ including javax.json into pgjdbc.jar
>>>
>>> Álvaro , Can you please tell us if "using compile type dependency for both
>>> javax.json and RI" suits you?
>>>
>> Vladimir
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 30/06/15 19:33, Steven Schlansker wrote:
> On Jun 30, 2015, at 10:25 AM, Álvaro Hernández Tortosa <aht@8Kdata.com> wrote:
>
>>     Thanks for asking for the double-check. No, indeed I'm still asking to provide the class files for the API in
thepackage. I feel that's the right way, and I don't see it would create conflicts unless JSR353 would create a new
version,something which I believe extremely unlikely until it merges with Java10 or JDBC5 comes out, point at which we
wouldneed to change things anyway. 
> I do not believe this is as unlikely as you think.  For example, the javax.ws.rs JAX-RS spec has 22 different
versionsavailable on Maven Central.  Granted, many of these are milestone builds and not releases, but there are
alreadytwo major versions, a patch available and a new minor version coming through the pipes. 
>
> So assuming these spec jars will not change is provably false.  One of my fears is that if PG bundles JSR353 1.0, and
Ineed JSR353 1.0.1, I now need to fight the driver packaging to upgrade a theoretically unrelated package. 

     So this is a matter of beliefs and probabilities :)

     I believe it will not be changed before JDBC5/Java10. And by then,
we would need to change everything anyway. I may be wrong, of course.

     So let's do a cost analysis: if I were right, we would have done
the best thing and we would have provided the most user friendly
solution. If I am not, when that happens (and we will have enough time,
since JSR is an open process), we will need to provide a new driver
version, perform the split into two packages and write all the READMEs
to explain users where/how to get the dependencies.

     If, instead, we opt now for betting on JSR353 evolution, we would
need to create a new driver version, perform the split into packages and
write the READMEs.

     In other words: if my bet is correct, we save some work and make
life easier for the users. If I am wrong, we will end up doing the same
work --although later on.

     So unless I am missing something, cost analysis IMHO suggests to
bet on not having to do the change ;)

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



>
>>     However, I don't want to insist more or suck more dev bandwitch, that's my opinion and it's been stated more
timesthan I wish, so I would now leave the decision to the rest of you :) 
>>
>>     Regards,
>>
>>     Alvaro
>>
>>
>> --
>> Álvaro Hernández Tortosa
>>
>>
>> -----------
>> 8Kdata
>>
>>
>>
>> On 30/06/15 18:49, Vladimir Sitnikov wrote:
>>> ah. I meant to double-check with Álvaro if he is suggesting compile
>>> type dependency.
>>>
>>> If he means that we in fact are discussing the same thing, so no
>>> contradiction exists.
>>>
>>>> However, regarding POLA you say "compile dependency" which means you
>>>> suggest _not_ including javax.json into pgjdbc.jar
>>>>
>>>> Álvaro , Can you please tell us if "using compile type dependency for both
>>>> javax.json and RI" suits you?
>>>>
>>> Vladimir
>>
>>
>> --
>> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
"Markus KARG"
Date:
Being a JAX-RS EG member I need to chime in here: JAX-RS actually is fully
backwards compatible, fully intentionally. We will never break any existing
feature, and I think I am speaking for hardly every JSR EG when I say that
most (if not all) JSR standards are always backwards compatible. The 22
different versions actually are no releases but development milestones not
intended for productive use.

All parts of Java EE have a restriction that nothing may break backwards
compatibility ever. This not only is true of JAX-RS but most certainly also
JSON-P and JSON-B. It is a guarantee the Java EE spec lead (Bill Shannon)
gives for all parts of the Java EE platform. So we can really safely rely on
that.

-Markus

On Jun 30, 2015, at 10:25 AM, Álvaro Hernández Tortosa <aht@8Kdata.com>
wrote:

>
>    Thanks for asking for the double-check. No, indeed I'm still asking to
provide the class files for the API in the package. I feel that's the right
way, and I don't see it would create conflicts unless JSR353 would create a
new version, something which I believe extremely unlikely until it merges
with Java10 or JDBC5 comes out, point at which we would need to change
things anyway.

I do not believe this is as unlikely as you think.  For example, the
javax.ws.rs JAX-RS spec has 22 different versions available on Maven
Central.  Granted, many of these are milestone builds and not releases, but
there are already two major versions, a patch available and a new minor
version coming through the pipes.

So assuming these spec jars will not change is provably false.  One of my
fears is that if PG bundles JSR353 1.0, and I need JSR353 1.0.1, I now need
to fight the driver packaging to upgrade a theoretically unrelated package.

>
>    However, I don't want to insist more or suck more dev bandwitch, that's
my opinion and it's been stated more times than I wish, so I would now leave
the decision to the rest of you :)
>
>    Regards,
>
>    Alvaro
>
>
> --
> Álvaro Hernández Tortosa
>
>
> -----------
> 8Kdata
>
>
>
> On 30/06/15 18:49, Vladimir Sitnikov wrote:
>> ah. I meant to double-check with Álvaro if he is suggesting compile
>> type dependency.
>>
>> If he means that we in fact are discussing the same thing, so no
>> contradiction exists.
>>
>>> However, regarding POLA you say "compile dependency" which means you
>>> suggest _not_ including javax.json into pgjdbc.jar
>>>
>>> Álvaro , Can you please tell us if "using compile type dependency for
both
>>> javax.json and RI" suits you?
>>>
>> Vladimir
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc




Re: SQLJSON

From
Steven Schlansker
Date:
Sorry, maybe I'm misunderstanding:  my complaint is that if the PG driver packages
version 1.0, but I use a feature added in a hypothetical 1.1, then the 1.0 bundled in
PG becomes a problem for me.

So what I am talking about is *forward* compatibility, not backward.

I'm not claiming this is a huge issue, I just wanted to point out that these
interfaces can and do evolve, so it is a false assumption that the version
baked into the PG driver will be good for all time.


On Jun 30, 2015, at 1:04 PM, Markus KARG <markus@headcrashing.eu> wrote:

> Being a JAX-RS EG member I need to chime in here: JAX-RS actually is fully
> backwards compatible, fully intentionally. We will never break any existing
> feature, and I think I am speaking for hardly every JSR EG when I say that
> most (if not all) JSR standards are always backwards compatible. The 22
> different versions actually are no releases but development milestones not
> intended for productive use.
>
> All parts of Java EE have a restriction that nothing may break backwards
> compatibility ever. This not only is true of JAX-RS but most certainly also
> JSON-P and JSON-B. It is a guarantee the Java EE spec lead (Bill Shannon)
> gives for all parts of the Java EE platform. So we can really safely rely on
> that.
>
> -Markus
>
> On Jun 30, 2015, at 10:25 AM, Álvaro Hernández Tortosa <aht@8Kdata.com>
> wrote:
>
>>
>>   Thanks for asking for the double-check. No, indeed I'm still asking to
> provide the class files for the API in the package. I feel that's the right
> way, and I don't see it would create conflicts unless JSR353 would create a
> new version, something which I believe extremely unlikely until it merges
> with Java10 or JDBC5 comes out, point at which we would need to change
> things anyway.
>
> I do not believe this is as unlikely as you think.  For example, the
> javax.ws.rs JAX-RS spec has 22 different versions available on Maven
> Central.  Granted, many of these are milestone builds and not releases, but
> there are already two major versions, a patch available and a new minor
> version coming through the pipes.
>
> So assuming these spec jars will not change is provably false.  One of my
> fears is that if PG bundles JSR353 1.0, and I need JSR353 1.0.1, I now need
> to fight the driver packaging to upgrade a theoretically unrelated package.
>
>>
>>   However, I don't want to insist more or suck more dev bandwitch, that's
> my opinion and it's been stated more times than I wish, so I would now leave
> the decision to the rest of you :)
>>
>>   Regards,
>>
>>   Alvaro
>>
>>
>> --
>> Álvaro Hernández Tortosa
>>
>>
>> -----------
>> 8Kdata
>>
>>
>>
>> On 30/06/15 18:49, Vladimir Sitnikov wrote:
>>> ah. I meant to double-check with Álvaro if he is suggesting compile
>>> type dependency.
>>>
>>> If he means that we in fact are discussing the same thing, so no
>>> contradiction exists.
>>>
>>>> However, regarding POLA you say "compile dependency" which means you
>>>> suggest _not_ including javax.json into pgjdbc.jar
>>>>
>>>> Álvaro , Can you please tell us if "using compile type dependency for
> both
>>>> javax.json and RI" suits you?
>>>>
>>> Vladimir
>>
>>
>>
>> --
>> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-jdbc
>
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
"Markus KARG"
Date:
Understood. That's why I am deeply against any kind of bundling. :-)


-----Original Message-----
From: Steven Schlansker [mailto:stevenschlansker@gmail.com]
Sent: Dienstag, 30. Juni 2015 22:17
To: Markus KARG
Cc: List
Subject: Re: [JDBC] SQLJSON

Sorry, maybe I'm misunderstanding:  my complaint is that if the PG driver
packages version 1.0, but I use a feature added in a hypothetical 1.1, then
the 1.0 bundled in PG becomes a problem for me.

So what I am talking about is *forward* compatibility, not backward.

I'm not claiming this is a huge issue, I just wanted to point out that these
interfaces can and do evolve, so it is a false assumption that the version
baked into the PG driver will be good for all time.


On Jun 30, 2015, at 1:04 PM, Markus KARG <markus@headcrashing.eu> wrote:

> Being a JAX-RS EG member I need to chime in here: JAX-RS actually is
> fully backwards compatible, fully intentionally. We will never break
> any existing feature, and I think I am speaking for hardly every JSR
> EG when I say that most (if not all) JSR standards are always
> backwards compatible. The 22 different versions actually are no
> releases but development milestones not intended for productive use.
>
> All parts of Java EE have a restriction that nothing may break
> backwards compatibility ever. This not only is true of JAX-RS but most
> certainly also JSON-P and JSON-B. It is a guarantee the Java EE spec
> lead (Bill Shannon) gives for all parts of the Java EE platform. So we
> can really safely rely on that.
>
> -Markus
>
> On Jun 30, 2015, at 10:25 AM, Álvaro Hernández Tortosa
> <aht@8Kdata.com>
> wrote:
>
>>
>>   Thanks for asking for the double-check. No, indeed I'm still asking
>> to
> provide the class files for the API in the package. I feel that's the
> right way, and I don't see it would create conflicts unless JSR353
> would create a new version, something which I believe extremely
> unlikely until it merges with Java10 or JDBC5 comes out, point at
> which we would need to change things anyway.
>
> I do not believe this is as unlikely as you think.  For example, the
> javax.ws.rs JAX-RS spec has 22 different versions available on Maven
> Central.  Granted, many of these are milestone builds and not
> releases, but there are already two major versions, a patch available
> and a new minor version coming through the pipes.
>
> So assuming these spec jars will not change is provably false.  One of
> my fears is that if PG bundles JSR353 1.0, and I need JSR353 1.0.1, I
> now need to fight the driver packaging to upgrade a theoretically
unrelated package.
>
>>
>>   However, I don't want to insist more or suck more dev bandwitch,
>> that's
> my opinion and it's been stated more times than I wish, so I would now
> leave the decision to the rest of you :)
>>
>>   Regards,
>>
>>   Alvaro
>>
>>
>> --
>> Álvaro Hernández Tortosa
>>
>>
>> -----------
>> 8Kdata
>>
>>
>>
>> On 30/06/15 18:49, Vladimir Sitnikov wrote:
>>> ah. I meant to double-check with Álvaro if he is suggesting compile
>>> type dependency.
>>>
>>> If he means that we in fact are discussing the same thing, so no
>>> contradiction exists.
>>>
>>>> However, regarding POLA you say "compile dependency" which means
>>>> you suggest _not_ including javax.json into pgjdbc.jar
>>>>
>>>> Álvaro , Can you please tell us if "using compile type dependency
>>>> for
> both
>>>> javax.json and RI" suits you?
>>>>
>>> Vladimir
>>
>>
>>
>> --
>> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make
>> changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-jdbc
>
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org) To make
> changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc




Re: SQLJSON

From
Vitalii Tymchyshyn
Date:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:

    To you and I, we know that JsonValue is a JSR353 standard and hence a external dependency and hence an action to add to the classpath is required.

    A given user might not be aware that JsonValue is a standard, and it's reasonable to think it is a Pg-specific class and thus you'd expect it to be bundled with the driver. Indeed, it sounds weird that you need extra dependencies to process objects returned by the driver. I'd always expect the driver to contain all the code to work with it!

    If JsonValue would be an own class, would we also tell the user "grab an external dependency too"? I don't think so. Then, why does the user need to care about that? I think this should be abstracted away for the user.


    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata


On 01/07/15 00:30, Vitalii Tymchyshyn wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Dave Cramer
Date:
Do we really think there will be an update to JSR 353 ??

if not then there is little harm in bundling it

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
John R Pierce
Date:
On 6/30/2015 3:30 PM, Vitalii Tymchyshyn wrote:
>
> I dont understand how users can "not realize they need another
> dependency" if their application is going to use Json.
> How they think they are going to use it without having it in classpath?
>
> Best regards, Vitalii Tymchyshyn
>

what about all of us who don't use JSON in our databases as it really is
a poor fit to the relational calculus ?   if there are dependencies on
modules that aren't part of J2SE, we'll be rather annoyed.




--
john r pierce, recycling bits in santa cruz



Re: SQLJSON

From
Álvaro Hernández Tortosa
Date:
On 01/07/15 00:42, John R Pierce wrote:
> On 6/30/2015 3:30 PM, Vitalii Tymchyshyn wrote:
>>
>> I dont understand how users can "not realize they need another
>> dependency" if their application is going to use Json.
>> How they think they are going to use it without having it in classpath?
>>
>> Best regards, Vitalii Tymchyshyn
>>
>
> what about all of us who don't use JSON in our databases as it really
> is a poor fit to the relational calculus ?   if there are dependencies
> on modules that aren't part of J2SE, we'll be rather annoyed.

     May I ask why would you be annoyed by a 32Kb or 96Kb worst case
size increase in the driver? If you don't use it, that's fine. BTW, you
can map JSON to relational.... check out ToroDB (www.torodb.com).
Disclaimer: I'm a ToroDB dev :)

     Regards,

     Álvaro

--
Álvaro Hernández Tortosa


-----------
8Kdata



Re: SQLJSON

From
Vitalii Tymchyshyn
Date:
As for me it should not be required unless user explicitly asks for it with two-params getObject. Single param must not be changed as this would break backward compatibility. Or it can be controlled with a uri param, but I'd say default should be to return String until after new version of JDBC says otherwise.

Actually, I'd vote for pluggable converters for any class for two-param getObject. For JSON some of them could be prebundled, like for JSR and Jackson. Note that JSR or Jackson itself should not be included.
Also user should be able to write his own converter and to specify what to use by default for single-param getObject.

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 о 18:36 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

    To you and I, we know that JsonValue is a JSR353 standard and hence a external dependency and hence an action to add to the classpath is required.

    A given user might not be aware that JsonValue is a standard, and it's reasonable to think it is a Pg-specific class and thus you'd expect it to be bundled with the driver. Indeed, it sounds weird that you need extra dependencies to process objects returned by the driver. I'd always expect the driver to contain all the code to work with it!

    If JsonValue would be an own class, would we also tell the user "grab an external dependency too"? I don't think so. Then, why does the user need to care about that? I think this should be abstracted away for the user.



    Regards,

    Álvaro


-- 
Álvaro Hernández Tortosa


-----------
8Kdata


On 01/07/15 00:30, Vitalii Tymchyshyn wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Re: SQLJSON

From
Vladimir Sitnikov
Date:
>but I'd say default should be to return String until after new version of JDBC says otherwise.

+1

>Actually, I'd vote for pluggable converters for any class for two-param getObject. For JSON some of them could be
prebundled,like for JSR >and Jackson. Note that JSR or Jackson itself should not be included. 
>Also user should be able to write his own converter and to specify what to use by default for single-param getObject

+1

Vladimir


Re: SQLJSON

From
Christopher BROWN
Date:
Hello,

JSR 353 does not address things like JSON schemas (there is no standard at this time, but this may change, and there are many non-standard implementations) or binary encoding formats (again, no current standard, but several implementations).  So there is scope for the API to change, with forward-compatibility issues especially if there are classloader conflicts.

Bundling the API by default may indeed cause harm if the application already has JSONP in the classpath (highly likely in Java EE environments, and already the case in many OSGi environments).  I wouldn't mind making things easy for users by bundling it if it had no side effects for other users, but it WILL, and that'll happen with the current API, not just potential forward-compatibility issues.  It's not a neutral packaging issue, and those that are against it are not just motivated by ivory-tower engineering.  Indeed, instead of a clear linkage error, users who don't read the famous manual and who don't really know about how classloading works, will have much greater difficulty understanding and resolving such problems.

As Markus informed us by forwarding Lance Andersen's opinion (he's the JDBC spec lead):

...it is expected that the client application provide the classes, not the driver.  This can be done by overloading the getObject() methods (not the default one; if this is change, then the change should only be activated by a connection property to maintain backwards compatibility).  Furthermore, in the case of injected drivers (JNDI, Spring, other pools), you won't even have PostgreSQL in the classpath, so to use JSONP, you'll need to add the API in your classpath... but because your JSONP classes are not loaded from the same source as the driver-bundled ones, the JVM will consider them as different, and it just won't work, even with identical class names.

So, why deviate from JDBC standard practice, why trip up those that already have an application server or an imposed enterprise framework providing such classes, why make life difficult for those who actually read the documentation (I'm talking about just the "getting started" and possibly even "how to define a PostgreSQL JDBC URL...", not even dreaming of a developer reading beyond that), or even that category of users who just use brute force to stick a load of ".jars" together "until it works", when in fact for the reasons outlined above (and well-stated by many other posters here) even with such good intentions, for a large proportion of such users, it in fact won't just work?

I understand the intention, but there's a lot of real-world feedback here explaining why, except in the most trivial of situation, it won't just hurt "smart" developers, but also hurt a large proportion of those it is intended to help?

--
Christopher


On 1 July 2015 at 00:38, Dave Cramer <pg@fastcrypt.com> wrote:
Do we really think there will be an update to JSR 353 ??

if not then there is little harm in bundling it

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc


Re: SQLJSON

From
Dave Cramer
Date:
Yes, it would appear there is a preponderance of evidence that including the API and the RI in the driver would be a bad thing. It's my opinion that this would not be a good idea.

Thanks everyone for your opinions and patience discussing this thread.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 1 July 2015 at 03:15, Christopher BROWN <brown@reflexe.fr> wrote:
Hello,

JSR 353 does not address things like JSON schemas (there is no standard at this time, but this may change, and there are many non-standard implementations) or binary encoding formats (again, no current standard, but several implementations).  So there is scope for the API to change, with forward-compatibility issues especially if there are classloader conflicts.

Bundling the API by default may indeed cause harm if the application already has JSONP in the classpath (highly likely in Java EE environments, and already the case in many OSGi environments).  I wouldn't mind making things easy for users by bundling it if it had no side effects for other users, but it WILL, and that'll happen with the current API, not just potential forward-compatibility issues.  It's not a neutral packaging issue, and those that are against it are not just motivated by ivory-tower engineering.  Indeed, instead of a clear linkage error, users who don't read the famous manual and who don't really know about how classloading works, will have much greater difficulty understanding and resolving such problems.

As Markus informed us by forwarding Lance Andersen's opinion (he's the JDBC spec lead):

...it is expected that the client application provide the classes, not the driver.  This can be done by overloading the getObject() methods (not the default one; if this is change, then the change should only be activated by a connection property to maintain backwards compatibility).  Furthermore, in the case of injected drivers (JNDI, Spring, other pools), you won't even have PostgreSQL in the classpath, so to use JSONP, you'll need to add the API in your classpath... but because your JSONP classes are not loaded from the same source as the driver-bundled ones, the JVM will consider them as different, and it just won't work, even with identical class names.

So, why deviate from JDBC standard practice, why trip up those that already have an application server or an imposed enterprise framework providing such classes, why make life difficult for those who actually read the documentation (I'm talking about just the "getting started" and possibly even "how to define a PostgreSQL JDBC URL...", not even dreaming of a developer reading beyond that), or even that category of users who just use brute force to stick a load of ".jars" together "until it works", when in fact for the reasons outlined above (and well-stated by many other posters here) even with such good intentions, for a large proportion of such users, it in fact won't just work?

I understand the intention, but there's a lot of real-world feedback here explaining why, except in the most trivial of situation, it won't just hurt "smart" developers, but also hurt a large proportion of those it is intended to help?

--
Christopher


On 1 July 2015 at 00:38, Dave Cramer <pg@fastcrypt.com> wrote:
Do we really think there will be an update to JSR 353 ??

if not then there is little harm in bundling it

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:

On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc



Re: SQLJSON

From
Florent Guillaume
Date:
I totally agree that bundling any kind of external API or non-shaded
library inside the driver would be a very bad thing.

I work every day with a big open source application that has to
include close to 250 external libraries, and trust me anytime some
library tries to do something clever and include other parts of some
APIs that do not belong to its namespace, it's a world of pain. It
even includes libraries that "helpfully" include a log4j.xml,
simplelogger.properties or similar META-INF/services/*, all things
which DON'T belong in a library but in an application.

So please, don't bundle the JSON API or the RI or Jackson in the driver.

The *first time* the developer will try to use it without proper
dependencies he'll get an error message (the driver can provide
helpful errors here), he'll add the missing dependencies, and this
will be it. No big deal, and it keeps things cleanly separated.

Florent

On Wed, Jul 1, 2015 at 1:31 PM, Dave Cramer <pg@fastcrypt.com> wrote:
> Yes, it would appear there is a preponderance of evidence that including the
> API and the RI in the driver would be a bad thing. It's my opinion that this
> would not be a good idea.
>
> Thanks everyone for your opinions and patience discussing this thread.
>
> Dave Cramer
>
> dave.cramer(at)credativ(dot)ca
> http://www.credativ.ca
>
> On 1 July 2015 at 03:15, Christopher BROWN <brown@reflexe.fr> wrote:
>>
>> Hello,
>>
>> JSR 353 does not address things like JSON schemas (there is no standard at
>> this time, but this may change, and there are many non-standard
>> implementations) or binary encoding formats (again, no current standard, but
>> several implementations).  So there is scope for the API to change, with
>> forward-compatibility issues especially if there are classloader conflicts.
>>
>> Bundling the API by default may indeed cause harm if the application
>> already has JSONP in the classpath (highly likely in Java EE environments,
>> and already the case in many OSGi environments).  I wouldn't mind making
>> things easy for users by bundling it if it had no side effects for other
>> users, but it WILL, and that'll happen with the current API, not just
>> potential forward-compatibility issues.  It's not a neutral packaging issue,
>> and those that are against it are not just motivated by ivory-tower
>> engineering.  Indeed, instead of a clear linkage error, users who don't read
>> the famous manual and who don't really know about how classloading works,
>> will have much greater difficulty understanding and resolving such problems.
>>
>> As Markus informed us by forwarding Lance Andersen's opinion (he's the
>> JDBC spec lead):
>> http://www.postgresql.org/message-id/007701d0b2b6$4cc930b0$e65b9210$@eu
>>
>> ...it is expected that the client application provide the classes, not the
>> driver.  This can be done by overloading the getObject() methods (not the
>> default one; if this is change, then the change should only be activated by
>> a connection property to maintain backwards compatibility).  Furthermore, in
>> the case of injected drivers (JNDI, Spring, other pools), you won't even
>> have PostgreSQL in the classpath, so to use JSONP, you'll need to add the
>> API in your classpath... but because your JSONP classes are not loaded from
>> the same source as the driver-bundled ones, the JVM will consider them as
>> different, and it just won't work, even with identical class names.
>>
>> So, why deviate from JDBC standard practice, why trip up those that
>> already have an application server or an imposed enterprise framework
>> providing such classes, why make life difficult for those who actually read
>> the documentation (I'm talking about just the "getting started" and possibly
>> even "how to define a PostgreSQL JDBC URL...", not even dreaming of a
>> developer reading beyond that), or even that category of users who just use
>> brute force to stick a load of ".jars" together "until it works", when in
>> fact for the reasons outlined above (and well-stated by many other posters
>> here) even with such good intentions, for a large proportion of such users,
>> it in fact won't just work?
>>
>> I understand the intention, but there's a lot of real-world feedback here
>> explaining why, except in the most trivial of situation, it won't just hurt
>> "smart" developers, but also hurt a large proportion of those it is intended
>> to help?
>>
>> --
>> Christopher
>>
>>
>> On 1 July 2015 at 00:38, Dave Cramer <pg@fastcrypt.com> wrote:
>>>
>>> Do we really think there will be an update to JSR 353 ??
>>>
>>> if not then there is little harm in bundling it
>>>
>>> Dave Cramer
>>>
>>> dave.cramer(at)credativ(dot)ca
>>> http://www.credativ.ca
>>>
>>> On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:
>>>>
>>>> I dont understand how users can "not realize they need another
>>>> dependency" if their application is going to use Json.
>>>> How they think they are going to use it without having it in classpath?
>>>>
>>>> Best regards, Vitalii Tymchyshyn
>>>>
>>>>
>>>> Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:
>>>>>
>>>>>
>>>>> On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>>>> >>   so that you don't need to parse it manually, which is a pain for
>>>>> >> many users
>>>>> > A) Can you please list pain points in literal?
>>>>>
>>>>>      I did that already:
>>>>> http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com
>>>>>
>>>>>      In brief: users don't realize they need another dependency. Then
>>>>> they find their code breaks with a RuntimeException. Frustration
>>>>> appears, and there is no compelling reason not to have the dependency
>>>>> in
>>>>> the driver. User gets angry. I agree with the user anger.
>>>>>
>>>>>      Imagine JSR353 wasn't an external class. Imagine the driver wanted
>>>>> to return a PGJsonValue class. Now you compile the code and then remove
>>>>> the .class from the JAR file. The end result is more or less the same:
>>>>> hey, to use this driver you have to anyway include another dependency.
>>>>> Which is weird: a dependency on something you are returning......
>>>>>
>>>>>      But again: *what* is the problem adding this dependency? Classpath
>>>>> collision is extremely unlikely, I won't bet on it, so why not help the
>>>>> user even more? Why not make it more user friendly, why do we prefer
>>>>> less user friendly?
>>>>>
>>>>>
>>>>> > I think we've already agreed that "having readme" is good enough so
>>>>> > even newbies can add dependencies.
>>>>>      I don't agree is good enough. Good enough is not needed to add
>>>>> anything else. Or not asking the user to do something we could have
>>>>> provided instead with 0 drawbacks.
>>>>>
>>>>> > What are the other pain points except "need to add a couple of jars
>>>>> > if
>>>>> > json is needed"?
>>>>> >
>>>>> > Frankly speaking, I find current "different jdbc version jars" even
>>>>> > more confusing than "if you want json, add pgjdbc-json dependency".
>>>>>
>>>>>      Multiple-jars would be for experienced user who would like to
>>>>> override the RI. Everybody else would be happy with the default one. I
>>>>> wouldn't call it "multiple JARs", just an "expert" JARs for those who
>>>>> have that unfrequent needs.
>>>>>
>>>>> >
>>>>> >> so that you don't need to parse it manually,
>>>>> > B) That does not work that way.
>>>>> > Can you suggest a piece of "user code" you have in mind?
>>>>> >
>>>>> > Let me predict it a bit:
>>>>> > 1) It processes resultset
>>>>> > 2) Fetches "JsonValue"
>>>>> > 3) Passes to a business-logic class
>>>>> >
>>>>> > We can stop here.
>>>>> > "business logic class" from #3 will definitely have "JsonValue"
>>>>> > classes in its method signatures.
>>>>>
>>>>>      No, it doesn't need to be like that. That you extract a JSON from
>>>>> your database, it doesn't mean that you have to have a JSON on your
>>>>> business logic. Not at all.
>>>>>
>>>>>
>>>>> > In other words, it would require to have "javax.json" API in the
>>>>> > class
>>>>> > path to compile.
>>>>> >
>>>>> > Then I would love to know how is the user addressing that without
>>>>> > adding "javax.json" API dependency to her build system of choice.
>>>>> >
>>>>> > The same business logic should be unit-tested (without DB), shouldn't
>>>>> > it?
>>>>> > That results in "it should be testable without pgdjbc dependency".
>>>>>
>>>>>      As I said, it doesn't need to. So if you *also* have JSON at your
>>>>> business-logic then yes, you need the API on your classpath. But that's
>>>>> obvious :)
>>>>> >
>>>>> > C) If we continue creating bundles here and there (not sure what is
>>>>> > "non offensive" wording here), the number of possible bundles would
>>>>> > explode exponentially. jdbc driver is not an application for
>>>>> > end-user,
>>>>> > so I do not see much reason why it should "bundle optional
>>>>> > dependencies without shading them".
>>>>> >
>>>>> > For instance, if we decide to use objectweb.asm bytecode transformer,
>>>>> > it would be perfectly fine to shade it into org.postgresql.asm...
>>>>> > package, so it is not exposed to public.
>>>>> >
>>>>> > However, with javax.json you cannot do package renaming since it
>>>>> > would
>>>>> > make no sense.
>>>>>
>>>>>      Right. So there's one bundle (JDBC+API+shaded RI) and another just
>>>>> with JDBC, for people that don't want Json and want to save 96Kb on
>>>>> their Jars or "expert" people that want to override the RI. That's it.
>>>>> 2. That's not an exponential explosion.
>>>>>
>>>>> >
>>>>> > D) Theoretically, we could represent json as java.util.Map /
>>>>> > java.util.List by embedding and repackaging some json implementation,
>>>>> > however I am not sure if that makes much sense provided JSR353
>>>>> > defines
>>>>> > proper interfaces.
>>>>>      Yeah, I don't think that makes sense.
>>>>>
>>>>>      Regards,
>>>>>
>>>>>      Álvaro
>>>>>
>>>>>
>>>>> --
>>>>> Álvaro Hernández Tortosa
>>>>>
>>>>>
>>>>> -----------
>>>>> 8Kdata
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
>>>>> To make changes to your subscription:
>>>>> http://www.postgresql.org/mailpref/pgsql-jdbc
>>>
>>>
>>
>



--
Florent Guillaume, Director of R&D, Nuxeo
Open Source Content Management Platform for Business Apps
http://www.nuxeo.com   http://community.nuxeo.com


Re: SQLJSON

From
dmp
Date:
Christopher BROWN wrote:
> Hello,
>
> JSR 353 does not address things like JSON schemas (there is no standard at this
> time, but this may change, and there are many non-standard implementations) or
> binary encoding formats (again, no current standard, but several
> implementations).  So there is scope for the API to change, with
> forward-compatibility issues especially if there are classloader conflicts.
>
> Bundling the API by default may indeed cause harm if the application already has
> JSONP in the classpath (highly likely in Java EE environments, and already the
> case in many OSGi environments).  I wouldn't mind making things easy for users
> by bundling it if it had no side effects for other users, but it WILL, and
> that'll happen with the current API, not just potential forward-compatibility
> issues.  It's not a neutral packaging issue, and those that are against it are
> not just motivated by ivory-tower engineering.  Indeed, instead of a clear
> linkage error, users who don't read the famous manual and who don't really know
> about how classloading works, will have much greater difficulty understanding
> and resolving such problems.
>
> As Markus informed us by forwarding Lance Andersen's opinion (he's the JDBC spec
> lead):
> http://www.postgresql.org/message-id/007701d0b2b6$4cc930b0$e65b9210$@eu
>
> ...it is expected that the client application provide the classes, not the
> driver.  This can be done by overloading the getObject() methods (not the
> default one; if this is change, then the change should only be activated by a
> connection property to maintain backwards compatibility).  Furthermore, in the
> case of injected drivers (JNDI, Spring, other pools), you won't even have
> PostgreSQL in the classpath, so to use JSONP, you'll need to add the API in your
> classpath... but because your JSONP classes are not loaded from the same source
> as the driver-bundled ones, the JVM will consider them as different, and it just
> won't work, even with identical class names.
>
> So, why deviate from JDBC standard practice, why trip up those that already have
> an application server or an imposed enterprise framework providing such classes,
> why make life difficult for those who actually read the documentation (I'm
> talking about just the "getting started" and possibly even "how to define a
> PostgreSQL JDBC URL...", not even dreaming of a developer reading beyond that),
> or even that category of users who just use brute force to stick a load of
> ".jars" together "until it works", when in fact for the reasons outlined above
> (and well-stated by many other posters here) even with such good intentions, for
> a large proportion of such users, it in fact won't just work?
>
> I understand the intention, but there's a lot of real-world feedback here
> explaining why, except in the most trivial of situation, it won't just hurt
> "smart" developers, but also hurt a large proportion of those it is intended to
> help?
>
> --
> Christopher
>

+1

danap



Re: SQLJSON

From
"Markus KARG"
Date:

It is called JSR 374 and in progress already.

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Mittwoch, 1. Juli 2015 00:38
To: Vitalii Tymchyshyn
Cc: Álvaro Hernández Tortosa; Vladimir Sitnikov; List; Stephen Nelson
Subject: Re: [JDBC] SQLJSON

 

Do we really think there will be an update to JSR 353 ??

 

if not then there is little harm in bundling it


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

 

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:


On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 

Re: SQLJSON

From
"Markus KARG"
Date:

+1

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Vitalii Tymchyshyn
Sent: Mittwoch, 1. Juli 2015 02:02
To: Álvaro Hernández Tortosa; Vladimir Sitnikov
Cc: List; Stephen Nelson
Subject: Re: [JDBC] SQLJSON

 

As for me it should not be required unless user explicitly asks for it with two-params getObject. Single param must not be changed as this would break backward compatibility. Or it can be controlled with a uri param, but I'd say default should be to return String until after new version of JDBC says otherwise.

 

Actually, I'd vote for pluggable converters for any class for two-param getObject. For JSON some of them could be prebundled, like for JSR and Jackson. Note that JSR or Jackson itself should not be included.

Also user should be able to write his own converter and to specify what to use by default for single-param getObject.

 

Best regards, Vitalii Tymchyshyn

Вт, 30 черв. 2015 о 18:36 Álvaro Hernández Tortosa <aht@8kdata.com> пише:


    To you and I, we know that JsonValue is a JSR353 standard and hence a external dependency and hence an action to add to the classpath is required.

    A given user might not be aware that JsonValue is a standard, and it's reasonable to think it is a Pg-specific class and thus you'd expect it to be bundled with the driver. Indeed, it sounds weird that you need extra dependencies to process objects returned by the driver. I'd always expect the driver to contain all the code to work with it!

    If JsonValue would be an own class, would we also tell the user "grab an external dependency too"? I don't think so. Then, why does the user need to care about that? I think this should be abstracted away for the user.




    Regards,

    Álvaro



-- 
Álvaro Hernández Tortosa
 
 
-----------
8Kdata
 

 

On 01/07/15 00:30, Vitalii Tymchyshyn wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

 

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:


On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata



--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 

Re: SQLJSON

From
Gonzalo Ortiz Jaureguizar
Date:
I just read the JSR 374 page and it doesn't look like a JSR53 upgrade. It adds: new ways to modify JSON values (using builders) and creates methods on collection interface to generate a JSON value from them. It doesn't seem that JSR 374 changes JSR 353. In particular, it adds functionallity to work with JSR 353 JSON values, but does not modify them. In fact, the point 2.11 says:

2.11 Are there any existing specifications that might be rendered obsolete, deprecated, or in need of revision as a result of this work?

No.


2015-07-01 18:39 GMT+02:00 Markus KARG <markus@headcrashing.eu>:

It is called JSR 374 and in progress already.

 

 

From: pgsql-jdbc-owner@postgresql.org [mailto:pgsql-jdbc-owner@postgresql.org] On Behalf Of Dave Cramer
Sent: Mittwoch, 1. Juli 2015 00:38
To: Vitalii Tymchyshyn
Cc: Álvaro Hernández Tortosa; Vladimir Sitnikov; List; Stephen Nelson
Subject: Re: [JDBC] SQLJSON

 

Do we really think there will be an update to JSR 353 ??

 

if not then there is little harm in bundling it


Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

 

On 30 June 2015 at 18:30, Vitalii Tymchyshyn <vit@tym.im> wrote:

I dont understand how users can "not realize they need another dependency" if their application is going to use Json.
How they think they are going to use it without having it in classpath?

Best regards, Vitalii Tymchyshyn

 

Вт, 30 черв. 2015 07:34 Álvaro Hernández Tortosa <aht@8kdata.com> пише:


On 30/06/15 09:57, Vladimir Sitnikov wrote:
>>   so that you don't need to parse it manually, which is a pain for many users
> A) Can you please list pain points in literal?

     I did that already:
http://www.postgresql.org/message-id/558FC12E.8010107@8Kdata.com

     In brief: users don't realize they need another dependency. Then
they find their code breaks with a RuntimeException. Frustration
appears, and there is no compelling reason not to have the dependency in
the driver. User gets angry. I agree with the user anger.

     Imagine JSR353 wasn't an external class. Imagine the driver wanted
to return a PGJsonValue class. Now you compile the code and then remove
the .class from the JAR file. The end result is more or less the same:
hey, to use this driver you have to anyway include another dependency.
Which is weird: a dependency on something you are returning......

     But again: *what* is the problem adding this dependency? Classpath
collision is extremely unlikely, I won't bet on it, so why not help the
user even more? Why not make it more user friendly, why do we prefer
less user friendly?


> I think we've already agreed that "having readme" is good enough so
> even newbies can add dependencies.
     I don't agree is good enough. Good enough is not needed to add
anything else. Or not asking the user to do something we could have
provided instead with 0 drawbacks.

> What are the other pain points except "need to add a couple of jars if
> json is needed"?
>
> Frankly speaking, I find current "different jdbc version jars" even
> more confusing than "if you want json, add pgjdbc-json dependency".

     Multiple-jars would be for experienced user who would like to
override the RI. Everybody else would be happy with the default one. I
wouldn't call it "multiple JARs", just an "expert" JARs for those who
have that unfrequent needs.

>
>> so that you don't need to parse it manually,
> B) That does not work that way.
> Can you suggest a piece of "user code" you have in mind?
>
> Let me predict it a bit:
> 1) It processes resultset
> 2) Fetches "JsonValue"
> 3) Passes to a business-logic class
>
> We can stop here.
> "business logic class" from #3 will definitely have "JsonValue"
> classes in its method signatures.

     No, it doesn't need to be like that. That you extract a JSON from
your database, it doesn't mean that you have to have a JSON on your
business logic. Not at all.


> In other words, it would require to have "javax.json" API in the class
> path to compile.
>
> Then I would love to know how is the user addressing that without
> adding "javax.json" API dependency to her build system of choice.
>
> The same business logic should be unit-tested (without DB), shouldn't it?
> That results in "it should be testable without pgdjbc dependency".

     As I said, it doesn't need to. So if you *also* have JSON at your
business-logic then yes, you need the API on your classpath. But that's
obvious :)
>
> C) If we continue creating bundles here and there (not sure what is
> "non offensive" wording here), the number of possible bundles would
> explode exponentially. jdbc driver is not an application for end-user,
> so I do not see much reason why it should "bundle optional
> dependencies without shading them".
>
> For instance, if we decide to use objectweb.asm bytecode transformer,
> it would be perfectly fine to shade it into org.postgresql.asm...
> package, so it is not exposed to public.
>
> However, with javax.json you cannot do package renaming since it would
> make no sense.

     Right. So there's one bundle (JDBC+API+shaded RI) and another just
with JDBC, for people that don't want Json and want to save 96Kb on
their Jars or "expert" people that want to override the RI. That's it.
2. That's not an exponential explosion.

>
> D) Theoretically, we could represent json as java.util.Map /
> java.util.List by embedding and repackaging some json implementation,
> however I am not sure if that makes much sense provided JSR353 defines
> proper interfaces.
     Yeah, I don't think that makes sense.

     Regards,

     Álvaro


--
Álvaro Hernández Tortosa


-----------
8Kdata




--
Sent via pgsql-jdbc mailing list (pgsql-jdbc@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

 


Re: SQLJSON

From
Vladimir Sitnikov
Date:
>I just read

What does that mean?

Could you please explicitly cast +1 or -1 to Christopher's "I wouldn't
mind making things easy for users by bundling it if it had no side
effects for other users, but it WILL" mail?

Vladimir