Re: A new JDBC driver... - Mailing list pgsql-jdbc

<div>På tirsdag 12. mars 2013 kl. 02:19:11, skrev Kevin Wooten <<a href="mailto:kdubb@me.com"
target="_blank">kdubb@me.com</a>>:</div><blockquotestyle="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt
0pt0.8ex; padding-left: 1ex;">So… I did a crazy thing… I created an entirely new JDBC driver.<br /><br /> (For those
whocare there's an actual question at the bottom… this isn't just bragging)<br /><br /> About 10 days ago I set out to
fixthe current driver's support for composite and array objects.  Currently they are only ever retrieved as string
encodingdue to the use of the TEXT format for those types; parsing these encoded strings seemed just short of a
nightmare. Ambition got the better of me and before I knew it I had a "working" implementation of the entire FE/BE
protocol(save for Copy In/Out).<br /><br /> I began by hacking the driver to force it into binary mode and pulling the
rawdata with getBytes. Then I started building a framework for recognizing, encoding and decoding all the possible
typesPostgreSQL knows about.   Once that was working well, my mission was to rework this into the current driver.  This
provedalmost impossible due to 1) my limited familiarity with the code and 2) the assumptions it makes about the
formatsof things like parameters.  In the end it seemed too time consuming for ME to do this.  So, partly for fun, I
decidedto just implement the FE/BE protocol and see where it got me. Next thing I knew I was running queries and
retrievingdata.  Basically it's just a side project, of a retro-fit, that went wrong and has spiraled out of control
;)<br/><br /> Currently my "driver" (I use the term loosely as you can imagine it's state of compliance about 6 days of
realwork) looks like this:<br /><br /> * Basic support for JDBC interfaces (Driver, Connection, PreparedStatement,
ResultSet)<br/> * Can be discovered and loaded automatically through JDBC4 loader<br /> * Supports ONLY JDBC4
interfaces(no attempt to compile to only JDBC2/3 is made)<br /> * Lots of stuff missing, non-conformant or just plain
broken<br/> * Speaks "BINARY" format almost exclusively<br /> * Extremely clean and simple interface for supporting new
types;VERY DRY (only added in 1 place ever)<br /> * Unsupported formats can fallback to "TEXT"<br /> * Almost all of
thecurrent standard types are supported (even the hard ones like numerics, timestamps, dates)<br /> * Can decode any
recognizedtype to a Java object (this includes any imaginable composite or array type)<br /> * Connection.setTypeMap
andResultSet.get(int idx, Map) are both fully supported<br /> * Requests for composite objects that have no custom
mappingare returned as HashMap<br /> * Arrays can be decoded as a List, Map or native array (e.g. Object[], int[])<br
/>* As an an extension it can decode whole rows into POJO's as well (acts a tiny bit like MyBatis)<br /> * Asynchronous
I/Oengine provided by Netty<br /> * All connections share a single group of worker threads<br /> * LISTEN/NOTIFY and
notificationscan come through asynchronously<br /> * Netty has a great system for managing buffers and reading/writing
messagesthat shows increased speed<br /> * Performance wasn't a goal of this project but it's a nice side effect<br />
*Maven project<br /> * Written against Java 7<br /> * Obviously to enter any sort of real use the source version will
haveto be dialed down<br /> * Shows some serious gains in performance<br /> * Query and retrieve from the ResultSet a
millionrows with timestamps and it's about a 4-5x increase in speed<br /> * Obviously some types there is no speed
difference<br/> * Again I say, performance was NOT a goal of this project it's just a nice side effect<br /><br />
BINARYSUPPORT<br /> As outlined above I started the project to support decoding of Composite and Array types.  To
accomplishthis I download "pg_type", "pg_attribute" and "pg_proc" to the client upon connection.  I then create a type
registrythat holds all the required details of all the types.  Procedures, for both TEXT and BINARY protocols, are
lookedup and matched by name (e.g. "bool_send", "money_recv", etc) by a list of "Procedure Providers".  When a DataRow
messageis received it looks up the type in the registry and calls the appropriate TEXT or BINARY decoder to decode the
row. When sending parameter data the type is located and the it's encoder is called to encode the data. Reading through
thedriver mailing-lists, it seems using binary only has some ramifications as far as type coercion and such are
concerned;currently all user initiated queries use the Extended Protocol & Statement Describe to ensure parameter
types/valuesare correct.<br /><br /> Where to go from here…<br /><br /> The major question I would like to ask is… <br
/><br/> Should I continue on this path of a new driver and see if people join or should I take what I have learned and
tryto refit it into the current code?<br /><br /> I am no dummy.  I understand the years of experience the current
driverhas to ensure it works well in an extremely large number of cases.  At the same time, anybody who has peeked
aroundin there (and I have done quite a bit of it) knows its showing its age.   My driver is 100% new code… not a
stitchof the old was used.  Give this, it seems like transplanting my new "core" into the current project would be like
givingit a brain transplant just after a fresh head transplant; in other words… a rewrite.<br /><br /> I'd love it if
somefolks in the know could take a look at my code and see if it stirs up any ideas on integration or just makes you
wantto jump off a bridge.<br /><br /> If you read this far you get a cookie…<br /><br /> Here is the GitHub project…
 <ahref="https://github.com/kdubb/pgjdbc-ng" target="_blank">https://github.com/kdubb/pgjdbc-ng</a><br /><br />
Kevin</blockquote><div> </div><div>Coolio!!Can't wait to see what happens next. It's often fresh, ambitious, work like
thiswhich brings the world forward.</div><div> </div><div class="origo-email-signature">--<br /> Andreas Joseph Krogh
<andreak@officenet.no>     mob: +47 909 56 963<br /> Senior Software Developer / CTO - OfficeNet AS -
http://www.officenet.no<br/> Public key: http://home.officenet.no/~andreak/public_key.asc</div><div> </div>