Possibly of interest to driver developers and those with too much spare time..
I spent some time over the weekend playing with a Netty-based
reimplementation of the v3 protocol. It seems somewhat promising, the
code is a bit more manageable than the current v3 code so far and
using Netty to run the I/O in a separate thread could help with the
various deadlock / timeout / memory consumption issues that affect the
current driver.
For example, I could see batch execution working roughly like this:
* on the read side (upstream handler), buffer result rows as they
arrive, but turn off read interest when we have more than X amount of
data buffered
* on the write side, if we notice that write interest was turned off
(too much buffered write data) then:
* try to process some buffered results from the read side
* wait for write interest to be reasserted
which is hopefully deadlock-free regardless of the amount of result data.
It might help with the suggestions for processing result data as it
arrives rather than buffering it all first:
* upstream handler buffers result rows and turns off read interest
when more than X amount of data buffered
* ResultSet handler pulls data when needed from the buffer, which may
make us reassert read interest
* execution of a subsequent query forces the upstream handler to go
into "buffer everything" mode for any outstanding unconsumed execution
results
We can also do connection timeouts fairly cheaply without the current
horrible helper thread hack.
Patch is here: http://s3test.randomly.org/jdbc/0001-Skeleton-of-a-Netty-based-v3-protocol-implementation.patch
This gets as far as doing (MD5) authentication and processing startup
messages, then blows up because there's no query executor yet.
It also may have some odd things in there, as I'm using this as an
exercise to learn Netty & git at the same time, so YMMV.
Hopefully I'll have some more time to spend on this next weekend.
Oliver
