Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows) - Mailing list pgsql-hackers

From Robert Haas
Subject Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)
Date
Msg-id 603c8f070901051004w35b23e2j7c1c9d908faf0480@mail.gmail.com
Whole thread Raw
In response to Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  ("Douglas McNaught" <doug@mcnaught.org>)
Responses Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  (Andrew Chernow <ac@esilo.com>)
Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  (Gregory Stark <stark@enterprisedb.com>)
Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  ("Joshua D. Drake" <jd@commandprompt.com>)
List pgsql-hackers
> Are
> we willing to be dropped from Debian and possibly Red Hat if this is
> the case?

No.  I frankly think this discussion is a dead end.

The whole thing got started because Alex Hunsaker pointed out that his
database got a lot bigger because we disabled compression on columns >
1MB.  It seems like the obvious thing to do is turn it back on again.
The only objection to that is that it will hurt performance,
especially on substring operations.  That lead to a discussion of
alternative compression algorithms, which is only relevant if we
believe that there are people out there who want to do substring
extractions on huge columns AND want those columns to be compressed.
At least on this thread, we have zero requests for that feature
combination.

What we do have is a suggestion from several people that the database
shouldn't be in the business of compressing data AT ALL.  If we want
to implement that suggestion, then we could change the default column
storage type.

Regardless of whether we do that or not, no one has offered any
justification of the arbitrary decision not to compress columns >1MB,
and at least one person (Peter) has suggested that it is exactly
backwards.  I think he's right, and this part should be backed out.
That will leave us back in the sensible place where people who want
compression can get it (which is currently false) and people who don't
want it can get rid of it (which has always been true).  If there is a
demand for alternative compression algorithms, then someone can submit
a patch for that for 8.5.

...Robert


pgsql-hackers by date:

Previous
From: Stefan Kaltenbrunner
Date:
Subject: Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)
Next
From: Tom Lane
Date:
Subject: Re: version() output vs. 32/64 bits