Re: JDBC drivers and streaming content - Mailing list pgsql-hackers

From Dave Cramer
Subject Re: JDBC drivers and streaming content
Date
Msg-id 1043859914.1064.47.camel@inspiron.cramers
Whole thread Raw
In response to JDBC drivers and streaming content  ("Chris Smith" <chris@mindiq.com>)
List pgsql-hackers
Chris,

It's already being done, you should post this to the jdbc list.

Dave
On Fri, 2003-01-24 at 11:07, Chris Smith wrote:
> I'm about to start implemention streaming of queries to the server in the
> pgsql jdbc drivers when PreparedStatement is used with setBinaryStream...
> but before I get started, since I've never contributed before, I wanted to
> run it by everyone.
> 
> I'm planning on making the following changes:
> 
> 1. In QueryExecutor.sendQuery, if an object in m_binds is a java.io.Reader,
> then instead of calling toString and sending that, I'll read and send the
> contents in 16K increments.
> 
> 2. In AbstractJdbc1Statement.setBinaryStream, instead of delegating to
> setBytes, I'll call bind directly, but send a custom java.io.Reader subclass
> (which adds the quotes and properly escapes the bytes and such) as the
> object.
> 
> A couple questions:
> 
> - There are a few constants that could probably be tuned for performance
> (ie, the size of the buffer used for streaming, and a threshold to avoid the
> streaming overhead for very short streams).  Is there a fairly standard way
> to handle this stuff in the JDBC drivers?  Should it be made configurable to
> the user?  Read from a properties file?  Stored in any specific class as a
> constant?
> 
> - It seems to be quite a pain that org.postgresql.core.Encoding works only
> with String, and can't copy into a provided char[] -- this will mean
> creating a large number of String objects during the streaming.  I could fix
> this easily with java.nio.CharsetEncoder, but it would make the code
> dependent on JDK 1.4.  Not desired?
> 
> - If there's a general dislike for runtime testing of object classes among
> the developer community, then I could, instead of special-casing Reader in
> sendQuery, wrap *all* PreparedStatement parameters in an interface that has
> a getReader method, and provide a default non-streaming implementation that
> uses StringReader.  This is more intrusive, but I'll go whichever way makes
> it more likely for the patch to be committed.
> 
> - Am I missing anything?  Is this harder than it seems?  Seems like someone
> would have done it already...
> 
> --
> www.designacourse.com
> The Easiest Way to Train Anyone... Anywhere.
> 
> Chris Smith - Lead Software Developer/Technical Trainer
> MindIQ Corporation
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
-- 
Dave Cramer <dave@fastcrypt.com>
Cramer Consulting



pgsql-hackers by date:

Previous
From: "Dave Page"
Date:
Subject: Re: [mail] Re: Windows Build System
Next
From: Vince Vielhaber
Date:
Subject: Re: [mail] Re: Windows Build System