Re: psql and COPY BINARY - Mailing list pgsql-hackers

From Andreas Pflug
Subject Re: psql and COPY BINARY
Date
Msg-id 43A05736.6090208@pse-consulting.de
Whole thread Raw
In response to Re: psql and COPY BINARY  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Tom Lane wrote:
> Andreas Pflug <pgadmin@pse-consulting.de> writes:
> 
>>Tom Lane wrote:
>>
>>>There wasn't any obvious bang for the buck in rewriting it.
> 
> 
>>Well a non-binary copy could take as much as 5 times as much as a binary 
>>copy. I hit this when COPYing 1.5GB of data, getting a 6.6GB file. This 
>>made the 100MBit LAN connection a bottleneck.
> 
> 
> Or vice versa --- the binary format is *not* necessarily smaller than text.
> As an example, an integer column that contains only small values (say 1
> or 2 digits) will need 8 bytes as binary and only 2 or 3 as text.
> 
> Fixing psql to handle binary copy isn't an unreasonable thing to do,
> but I can't get real excited about it either ...

Having a choice can't be too bad.
A COMPRESSED option would be even better, but that's backend stuff (does 
TOAST use an algorithm that's platform independent?). Would have reduced 
the sample above to about 130MB.

Regards,
Andreas


pgsql-hackers by date:

Previous
From: Andreas Pflug
Date:
Subject: Re: Immodest Proposal: pg_catalog.pg_ddl
Next
From: Zoltan Boszormenyi
Date:
Subject: Re: Interesting speed anomaly