Re: bytea columns and large values - Mailing list pgsql-general

From Alban Hertroys
Subject Re: bytea columns and large values
Date
Msg-id CAF-3MvNrrFO94ERr93wid2fwaJ8TpXpMrARMsN=t2BHuE4yW_A@mail.gmail.com
Whole thread Raw
In response to Re: bytea columns and large values  (Radosław Smogura <rsmogura@softperience.eu>)
Responses Re: bytea columns and large values
List pgsql-general
On 29 September 2011 13:12, Radosław Smogura <rsmogura@softperience.eu> wrote:
>> sending ~1GB bytea values is borderline crazy, and is completely crazy
>> if you are not absolutely sure the transmission is not 100% binary.  I
>> don't know if the JDBC sends/receives bytea as binary, but it may not.
>>  If not, you might have better luck with the large object interface.

> "Crazy"? I thought that official stand was to keep such crazy values as
> TOAST, and LOB interface isn't something worth of usage.

Both are possible means to handle data-objects that large.

The difference between the two is that with BYTEA, the value in a
result-set is returned with the BYTEA value embedded, whereas with
LOBs you get a file-pointer that you can subsequently read out at your
leisure.

As a consequence, with objects of 1GB in size, BYTEA requires the
server to allocate over 1GB of memory for each record in the result
set until it can send such records to the client, while the memory
footprint with LOBs is MUCH smaller on the server-side; just a
file-handle.

The interface for LOBs is a little more complicated, due to getting a
file handle instead of directly receiving the large object, but you're
saving your server a mountain of memory-load.

> You have exposed such interface, and you firm it. Does this crazynies is
> measured by fixed amount above 500MB or it's fuzzy measured with standard
> deviation near 1GB? If I use bytea to store such values, looks I'm crazy
> too. Sorry, for thinking that documentation is trust worthy.

It gives you a choice. That doesn't automatically make it the best
choice for your situation. That's something only you can decide.

--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

pgsql-general by date:

Previous
From: Radosław Smogura
Date:
Subject: Re: bytea columns and large values
Next
From: Gregor Vollmer
Date:
Subject: Decimal vs. Bigint memory usage