Re: PostgreSQL 8.4 performance tuning questions - Mailing list pgsql-performance

From Merlin Moncure
Subject Re: PostgreSQL 8.4 performance tuning questions
Date
Msg-id b42b73150908050712xb51b8c9h623ac70d9ce843f5@mail.gmail.com
Whole thread Raw
In response to Re: PostgreSQL 8.4 performance tuning questions  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: PostgreSQL 8.4 performance tuning questions  (Scott Carey <scott@richrelevance.com>)
List pgsql-performance
On Tue, Aug 4, 2009 at 4:40 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> Scott Carey <scott@richrelevance.com> writes:
>> There are a handful of other compression algorithms very similar to LZO in
>> performance / compression level under various licenses.
>> LZO is just the best known and most widely used.
>
> And after we get done with the license question, we need to ask about
> patents.  The compression area is just a minefield of patents.  gzip is
> known to avoid all older patents (and would be pretty solid prior art
> against newer ones).  I'm far less confident about lesser-known systems.

I did a little bit of research.  LZO and friends are variants of LZW.
The main LZW patent died in 2003, and AFAIK there has been no patent
enforcement cases brought against LZO or it's cousins (LZO dates to
1996).  OK, I'm no attorney, etc, but the internet seems to believe
that the algorithms are patent free.  LZO is quite widely used, in
both open source and some relatively high profile commercial projects.

I downloaded the libraries and did some tests.
2.5 G sql dump:

compression time:
zlib: 4m 1s
lzo: 17s
fastlz: 28.8s
liblzf: 26.7s

compression size:
zlib: 609M 75%
lzo: 948M 62%
fastlz: 936M 62.5%
liblzf: 916M 63.5%

A couple of quick notes: liblzf produces (possibly) architecture
dependent archives according to its header, and fastlz is not declared
'stable' according to its website.

merlin

pgsql-performance by date:

Previous
From: Tom Lane
Date:
Subject: Re: GiST, caching, and consistency
Next
From: Scott Carey
Date:
Subject: Re: PostgreSQL 8.4 performance tuning questions