Re: Faster compression, again - Mailing list pgsql-hackers

From Daniel Farina
Subject Re: Faster compression, again
Date
Msg-id CAAZKuFZFfY0nz=ttAskKAUEs_0X=ORF9gpkYgWs9B=rVh-3x+Q@mail.gmail.com
Whole thread Raw
In response to Re: Faster compression, again  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Wed, Mar 14, 2012 at 2:58 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Daniel Farina <daniel@heroku.com> writes:
>> Given that, few I would say have had the traction that LZO and Snappy
>> have had, even though in many respects they are interchangeable in the
>> general trade-off spectrum. The question is: what burden of proof is
>> required to convince the project that Snappy does not have exorbitant
>> patent issues, in proportion to the utility of having a compression
>> scheme of this type integrated?
>
> Another not-exactly-trivial requirement is to figure out how to not
> break on-disk compatibility when installing an alternative compression
> scheme.  In hindsight it might've been a good idea if pglz_compress had
> wasted a little bit of space on some sort of version identifier ...
> but it didn't.

I was more thinking that the latency and throughput in LZ77 schemes
may be best first applied to protocol compression.  The downside is
that requires more protocol wrangling, but at least terabytes of
on-disk format doesn't get in the picture, even though LZ77 on the
data itself may be attractive.

I'm interested allowing layering transports below FEBE (similar to how
SSL is "below", except without the complexity of being tied into auth
& auth) in a couple of respects, and this is one of them.

--
fdr


pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: CREATE FOREGIN TABLE LACUNA
Next
From: Fabrízio de Royes Mello
Date:
Subject: Re: VALID UNTIL