Hi,
Coming back to another problem I have with the following insert statement:
(Postgres 7.4, driver build 215, jdbc 3)
ps.setCharacterStream(1, reader, (int) messageFile.length());
ps.executeUpdate();
The reason I do this (using a character stream to load a text file to a
TEXT field) is that I wish to avoid loading the file into memory completely.
I already noticed that the postgres driver does not stream the data to
the backend (I don't know if postgres actually supports this).
But what is worse, is that the file actually gets copied THREE times
into memory, causing my Java file to already get a java.lang.OutOfMemory
error with an insert of a 10M text file (Java allocates 20M for this,
since it has 2 byte chars, and the driver makes (at least) 3 copies
(referenced at the same time, not allowing one to be GC'ed), making 60M).
First of all, in setCharacterStream() it loads the file into a char array.
Then this is cloned into a String and passed to setString. (Here one can
already cause the char[] to go out of scope, at least allowing it to be
cleaned up). Secondly, setString causes another clone in escapeString().
(Ideally, one could read the stream into a StringBuffer big enough to allow
it to be escaped, and thereby only load the file into memory once instead
of three times). Finally, the driver (possibly) keeps large objects in
memory too long: for example, in setString() right after the bind, one can
already do a "sbuf.setLength(0)". This way, one does not have to wait
for setString (or another setter) to be called before the sbuf variable
is cleared.
Greetings,
Sebastiaan van Erk
