Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows) - Mailing list pgsql-hackers

From Robert Haas
Subject Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)
Date
Msg-id 603c8f070901051155vcb3e2cdgb8c1dd9323c733c5@mail.gmail.com
Whole thread Raw
In response to Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)  (Gregory Stark <stark@enterprisedb.com>)
List pgsql-hackers
On Mon, Jan 5, 2009 at 2:02 PM, Gregory Stark <stark@enterprisedb.com> wrote:
>> Regardless of whether we do that or not, no one has offered any
>> justification of the arbitrary decision not to compress columns >1MB,
>
> Er, yes, there was discussion before the change, for instance:
>
>  http://archives.postgresql.org/pgsql-hackers/2007-08/msg00082.php

OK, maybe I'm missing something, but I don't see anywhere in that
email where it suggests NEVER compressing anything above 1MB.  It
suggests some more nuanced things which are quite different.

> And do you have any response to this point?
>
>  I think the right value for this setting is going to depend on the
>  environment. If the system is starved for cpu cycles then you won't want to
>  compress large data. If it's starved for i/o bandwidth but has spare cpu
>  cycles then you will.
>
> http://archives.postgresql.org/pgsql-hackers/2009-01/msg00074.php

I think it is a good point, to the extent that compression is an
option that people choose in order to improve performance.  I'm not
really convinced that this is the case, but I haven't seen much
evidence on either side of the question.

> Well the original code had a threshold above which we *always* compresed even
> if it saved only a single byte.

I certainly don't think that's right either.

...Robert


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: EmitWarningsOnPlaceholders is too quiet
Next
From: Bruce Momjian
Date:
Subject: Re: Time to finalize patches for 8.4 beta