Re: Streaming bytea implementation offered - Mailing list pgsql-jdbc

From Werner Donné
Subject Re: Streaming bytea implementation offered
Date
Msg-id FC1BF631-720D-4E43-8234-B76415A86C24@pincette.biz
Whole thread Raw
In response to Re: Streaming bytea implementation offered  (Craig Ringer <ringerc@ringerc.id.au>)
Responses Re: Streaming bytea implementation offered  (Craig Ringer <ringerc@ringerc.id.au>)
List pgsql-jdbc
Op 01 Sep 2012 om 03:04 heeft Craig Ringer <ringerc@ringerc.id.au> het volgende geschreven:

> On 09/01/2012 12:49 AM, Werner Donné wrote:
>> Hello,
>>
>> I have modified version 9.1-902 of the driver in order for it to stream bytea values when calling
ResultSet.getBinaryStream().The implementation is here: 
>>
>> http://www.pincette.biz/home/werner/postgresql-jdbc-9.1-902-patched.src.zip
>>
>> When a bytea value is larger than 1MB it is streamed to a temporary file instead of being copied in the "answer"
array.When the stream is later accessed, either with "getBinaryInputStream" or "getBytes", the bytes are fetched from
thetemporary file, which is subsequently deleted. The "getBytes" method copies the bytes in the result array, while
"getBinaryInputStream"returns a FileInputStream with an overridden "close" method that deletes the temporary file 
>
> Sounds potentially good, but what happens if it's run inside the SecurityManager, or in an environment with no
abilityto create temporary files? 
>
> What about out-of-disk causing failures that wouldn't occur with OOM?

We will have to fall back on copying the byte array in case a temporary file couldn't be created, for whatever reason.

>
> Do you propose to make this always-on, or configurable?

The treashold could be configurable.

>
> How (if at all) does this interact with JDBC-standard BLOB support, and with large objects?

It has no effect on either.

>
>
> If you'd like this change considered, it'd be great if you could create a patch against the current PgJDBC sources,
ratherthan just publishing patched sources. I'd recommend that you clone PgJDBC (https://github.com/pgjdbc/pgjdbc): 
>
>    git://github.com/pgjdbc/pgjdbc.git
>
> then copy your changes into that working tree and use `git diff`. Alternately, you can , `git add` them, `git
commit`,then use `git format-patch` to get a diff; see 
>
>    http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html
>
> Make sure not to include any generated files, and please add some unit tests demonstrating the functionality if
possible.

I have never used git, nor have I written unit tests, but I'll have a look.

>
> --
> Craig Ringer
>

Werner.
--
http://www.pincette.biz/
Handling your documents with care, wherever you are.

pgsql-jdbc by date:

Previous
From: Craig Ringer
Date:
Subject: Re: Streaming bytea implementation offered
Next
From: Craig Ringer
Date:
Subject: Re: Streaming bytea implementation offered