Re: BLOB performance test FYI - Mailing list pgsql-jdbc

From Joe Shevland
Subject Re: BLOB performance test FYI
Date
Msg-id HEECIHEEJDBMCCGMGIOBAECLCHAA.jshevland@j-elite.com
Whole thread Raw
In response to Re: BLOB performance test FYI  (Anders Bengtsson <ndrsbngtssn@yahoo.se>)
Responses Re: BLOB performance test FYI  (Dave Cramer <Dave@micro-automation.net>)
Re: BLOB performance test FYI  (Anders Bengtsson <ndrsbngtssn@yahoo.se>)
List pgsql-jdbc
> > > I find that
> > > many users of bytea are storing gifs and html pages for web
> sites which
> > > generally are small enough such that bytea works fine.
> >
> > Well, I disagree... even a few kilobytes has shown to be substandard in
> > storing.
>
> Some of these problems aren't really about the datatypes themselves, but
> how the driver handles them.

Thats a good point; my issues have purely been through the JDBC driver interaction.

> Instead of streaming large data into the
> backend, the client reads them into memory before sending them.
> It seems the LO parts of the driver has a different solution, that
> streams the data directly.
> It would take some major reworking of the JDBC driver to fix this. I've
> looked at it, and concluded that it is possible, but that it was too big
> for me to fix.

I've got time (read: resigned a few weeks ago and am sick of hop-flavoured water ;) and wouldn't mind doing it; I just
neededan indication that its possible. Would you mind giving me some hints about what you looked at offline? 

I figured that the differences in the types and how the backend handles them precludes actually trying to do anything
tooptimise the behaviour in the client interface. For example, can I use the socket connection when performing an
updateoperation to stream binary data to the backend, or does it expect the whole statement to be character data sent
atthe one time? 

What about other client interfaces and bytea types, how does the TCP/IP stuff go? C, Perl et al would still have
problemsloading binary data into memory too I would have thought unless they stream the data. 

I'll add some notes re: this discussion to http://www.j-elite.com/pgprimer sometime today or tomorrow in the BLOB
section,then put aside some time to put some real content into the rest of the site. 

Cheers,
Joe


pgsql-jdbc by date:

Previous
From: Thomas O'Dowd
Date:
Subject: Re: using Date and setObject()
Next
From: Dave Cramer
Date:
Subject: Re: BLOB performance test FYI