Thread: data compression in protocol?

data compression in protocol?

From
Szymon Guz
Date:
Hi,
I've got a lot of databases, unfortunately applications need to get a lot of data (a lot of means too much). 
The whole request takes about 3s, while pure explain analyze only 250ms. 
The amount of downloaded data cannot be changed.

I'm just wondering if I can do anything about that.
Maybe there is some possibility of compressing the data in the protocol? Maybe some other way?

regards
Szymon

Re: data compression in protocol?

From
Grzegorz Jaśkiewicz
Date:
The answer is: SSL. SSL will compress things, before encrypting
(depends on setup obviously).
As far as I know, postgresql it self doesn't compress any data over the wire.


--
GJ

Re: data compression in protocol?

From
Merlin Moncure
Date:
2011/6/22 Grzegorz Jaśkiewicz <gryzman@gmail.com>:
> The answer is: SSL. SSL will compress things, before encrypting
> (depends on setup obviously).
> As far as I know, postgresql it self doesn't compress any data over the wire.

Could you elaborate a bit on using SSL for that purpose?  How do you
set it up for compression?  Can you set up openssl for no-op
encryption but to do compression?

merlin

Re: data compression in protocol?

From
Grzegorz Jaśkiewicz
Date:
http://httpd.apache.org/docs/2.0/ssl/ssl_faq.html#comp

That applies in general to SSL apps.

In cryptography it is always recommended, and sometimes even mandatory
to compress data before encryption. This reduces the risk of finding
patterns, etc.
And SSL includes that option as well.

But that's as far as my knowledge goes. Whether postgresql can utilize
it, and how to set it up with postgresql - dunno.


--
GJ

Re: data compression in protocol?

From
Craig Ringer
Date:
On 22/06/11 16:38, Szymon Guz wrote:
> Hi,
> I've got a lot of databases, unfortunately applications need to get a
> lot of data (a lot of means too much).
> The whole request takes about 3s, while pure explain analyze only 250ms.
> The amount of downloaded data cannot be changed.

What client are you using?

For libpq, as others have noted you can use OpenSSL-level compression.

For JDBC I think you're out of luck, as AFAIK the Java SSL
implementation doesn't support wire compression.

See this prior discussion on the topic:

http://postgresql.1045698.n5.nabble.com/Compression-on-SSL-links-td2261205.html

--
Craig Ringer