Thread: Re: [HACKERS] Binary Cursors, and the COPY command

(moved to -jdbc)

Thomas Hallgren wrote:

> Another more philosophical question (more suitable on the jdbc list) is when
> the Java 1.3 support should be limited (or perhaps discontinued altogether)
> so that further development can exploit everything that 1.4 provides. After
> all, it's been around for more than 2 years now. AFAIK, the early bugs
> forcing you to cling on to the 1.3 have been fixed a long time ago. Doesn't
> the current 3.0 driver make use of features from the Java 1.4 version of
> java.sql already?

JDBC3 support implies JDK1.4. But the driver has the JDBC3 support
separated out specifically so it's still possible to build under earlier
JDKs.

We only recently dropped JDK1.1 support (!). I think there might be some
resistance to dropping JDK1.2 / JDK1.3 support. Personally I target
JDK1.4 so it doesn't affect me anyway, but..

What features of 1.4 were you thinking of exploiting?

-O

Oliver Jowett wrote:

> What features of 1.4 were you thinking of exploiting?

Well, the java.nio obviously :-)

java.nio provides a java.nio.CharBuffer. A java.lang.StringBuffer is
synchronized. The CharBuffer is not. Since the JDBC driver uses strings
in a lot of places some code could be rewritten to increase performance.

And I think that Mark (pgsql@mohawksoft.com) has a point. 90% of all
installations would get a performance boost if native byte order was
used. IMHO, it's all about an initial protocoll negotiation (which of
course must be done in network byte order).

Regards,

Thomas Hallgren

Thomas Hallgren wrote:

> Well, the java.nio obviously :-)
>
> java.nio provides a java.nio.CharBuffer. A java.lang.StringBuffer is
> synchronized. The CharBuffer is not. Since the JDBC driver uses strings
> in a lot of places some code could be rewritten to increase performance.

I haven't seen the string manipulation to be much of a problem with the
current driver in the profiling I've done (it'll be quite application
specific though). And I'd have thought the stringbuffer monitors would
be essentially uncontended and cheap to enter. What are the hotspots you
see?

> And I think that Mark (pgsql@mohawksoft.com) has a point. 90% of all
> installations would get a performance boost if native byte order was
> used.

Well, not mine :) (x86 clients, sparc server, and the driver doesn't yet
use binary format in places where byte order matters anyway)

Also I am fairly suspicious about claims that native byte order will
make things go measurably faster. Do you have any profiling or
benchmarks to back that up? The low-level manipulation of protocol data
barely shows up on the profiles I've done.

-O

Oliver Jowett wrote:

> I haven't seen the string manipulation to be much of a problem with
> the current driver in the profiling I've done (it'll be quite
> application specific though). And I'd have thought the stringbuffer
> monitors would be essentially uncontended and cheap to enter. What are
> the hotspots you see?
>
See below...

> Also I am fairly suspicious about claims that native byte order will
> make things go measurably faster. Do you have any profiling or
> benchmarks to back that up? The low-level manipulation of protocol
> data barely shows up on the profiles I've done.

I haven't made any benchmarks and I admit that as long as the client and
server runs in separate processes, the gain will be relatively small.
Context switching and socket management are the real time consumers.

I'm the author of PL/Java. It uses its own JDBC driver on top of SPI.
The main reason for thas is that I don't want the overhead of streaming
data and flipping byte order when everything is readily available in
memory. When "client" and "server" resides in the same process the
overhead is measurable. By using java.nio in your JDBC, I beleive it
would be possible to not just use native byte ordering, but perhaps also
to create a nice abstraction allowing direct access to structures in
memory rather than streaming data, thus obliviate the need for my own
driver. And PL/Java will never run on Java 1.3 or older :-)

Do you have any opinion on that?

Regards,

Thomas Hallgren

Thomas Hallgren wrote:

> I'm the author of PL/Java. It uses its own JDBC driver on top of SPI.
> The main reason for thas is that I don't want the overhead of streaming
> data and flipping byte order when everything is readily available in
> memory. When "client" and "server" resides in the same process the
> overhead is measurable. By using java.nio in your JDBC, I beleive it
> would be possible to not just use native byte ordering, but perhaps also
> to create a nice abstraction allowing direct access to structures in
> memory rather than streaming data, thus obliviate the need for my own
> driver. And PL/Java will never run on Java 1.3 or older :-)
>
> Do you have any opinion on that?

This is a *lot* of work. There are bigger issues to deal with than the
use of NIO -- lots of the higher-level JDBC code makes assumptions about
the details of the protocol it is speaking. I've made some inroads on
that front with the V3 protocol rewrite I did recently but there will
still be a lot more work in that area.

That said, I can see that it might work with "SPI within the same
process" as a new underlying "protocol". That could use NIO or whatever
you want, and conditionally build that code only when 1.4 + the SPI
interface is available. The query/parameter abstraction that's currently
in place is intended to give the protocol flexibility about how it
represents the data (although it is pretty basic at the moment) so I
don't think it'd be necessary to have NIO in the protocol-independent
parts of the driver.

Anyway, good luck :)

-O