Re: long transfer time for binary data - Mailing list pgsql-general

From George Neuner
Subject Re: long transfer time for binary data
Date
Msg-id jgb6abla85trvb3vo6dqb57on8i1vvjo3u@4ax.com
Whole thread Raw
In response to long transfer time for binary data  (Johannes <jotpe@posteo.de>)
List pgsql-general
On Fri, 22 Jan 2016 22:05:24 +0100, Johannes <jotpe@posteo.de> wrote:

>Thanks for explanation. Im writing a client software in java/jdbc. Most
>images are in jpeg format. Some have high quality, most medium.

Unfortunately I'm not terribly conversant in Java ... I can
read/understand it, but I rarely write any.


>Rendering this 11MB Image in eog (Eye Of Gome) takes 0.5 sec, in GIMP it
>is very fast.

I'm not familiar with EoG, but GIMP uses both SIMD code to process the
image and OpenGL (which in turn uses your GPU if possible) to draw
directly to the video buffer.  That makes a big difference vs drawing
to a generic GUI window context.


>In Java the object createion takes nearly all time, the drawing is done
>very quickly.

I have zero experience with jdbc - but if it's anything like ODBC,
then it may be reading the images inefficiently (at least by default).
In ODBC connections have a settable MTU size - BLOBs that are bigger
than 1 MTU get transferred in pieces.

That is fine if you don't know the size of the object beforehand, but
it can be much slower than necessary if you do (or can approximate
it).  ODBC's default MTU is quite small by today's multimedia data
standards.

If it's something other than this - e.g., you need to process the
image faster from Java - then I'm afraid you'll have to look to
other's for help.


>The size of the binary string representation of this image is 22MB. I
>guess there are not other special transfer mechanism for binary data
>than plain text via sql, or?

You said originally it was a bytea column?  If so, the BLOB shouldn't
be any longer than the original image file.  It would be different if
you stored the image in a text column, e.g., as escaped ASCII or as
ROT64 encoded, etc.

Characters in Java are 16-bit values.  If you convert the BLOB into a
printable string [or your debugger does to view it], that string will
be twice as long as the binary.

Hope this helps,
George

pgsql-general by date:

Previous
From: John R Pierce
Date:
Subject: Re: 9.5 new features
Next
From: Vik Fearing
Date:
Subject: Re: 9.5 new features