Re: long transfer time for binary data - Mailing list pgsql-general

From Johannes
Subject Re: long transfer time for binary data
Date
Msg-id 56A68F13.2030203@posteo.de
Whole thread Raw
In response to Re: long transfer time for binary data  ("Daniel Verite" <daniel@manitou-mail.org>)
List pgsql-general
Am 25.01.2016 um 19:59 schrieb Daniel Verite:
>     Johannes wrote:
>
>> \lo_export 12345 /dev/null is completed in 0.86 seconds.
>
> If it's an 11MB file through a 100Mbits/s network, that's
> pretty much the best that can be expected.
>
> I would think the above is the baseline against which
> the other methods should be compared.
>
>> I sa my images as large object, which afaik is in practise not
>> readable with a binary cursor (we should use the lo_* functions). And of
>> course I already use the LargeObjectManager of the postgresql jdbc library.
>
> Sounds good.
>
>> You said, the server has to convert the bytes to hex string before
>> sending it over the wire.
>
> Only in certain contexts. SELECT lo_get(oid) is a query that returns
> bytea, so if the clients requests results in text format, they will
> be transferred as text, and it's the responsibility of the client
> to convert them back to bytes (or not, who knows, maybe the
> client wants hexadecimal).
>
> But lo_get is an exception, and besides a late addition (9.4 I believe).
> Generally, client-side access to large objects functions doesn't
> use a client-side SQL query, it's done through the
> "Function Call sub-protocol" described here:
> http://www.postgresql.org/docs/current/static/protocol-flow.html#AEN108750
> and the result comes back as binary.
>
> Presumably the JDBC LargeObjectManager uses that method.
>
> Best regards,

I thougth \lo_export can only run on server side only (like \copy copy).
0.8 seconds was the rutime on server to server disk.

Running from client (transfers only 12M):

real    0m3.386s
user    0m0.308s
sys    0m0.176s

Best regards Johannes


Attachment

pgsql-general by date:

Previous
From: "Joshua D. Drake"
Date:
Subject: Re: CoC [Final v2]
Next
From: Chris Travers
Date:
Subject: Re: CoC [Final v2]