Re: Force re-compression with lz4 - Mailing list pgsql-general

From Mladen Gogala
Subject Re: Force re-compression with lz4
Date
Msg-id d4ff3c0b-ac43-6b00-622a-501870fc2810@gmail.com
Whole thread Raw
In response to Force re-compression with lz4  (Florents Tselai <florents.tselai@gmail.com>)
Responses Re: Force re-compression with lz4
List pgsql-general
On 10/18/21 01:07, Michael Paquier wrote:
> CPU-speaking, LZ4 is*much*  faster than pglz when it comes to
> compression or decompression with its default options.  The
> compression ratio is comparable between both, still LZ4 compresses in
> average less than PGLZ.
> --
> Michael

LZ4 works much better with deduplication tools like Data Domain or Data 
Domain Boost (client side deduplication). With zip or gzip compression, 
deduplication ratios are much lower than with LZ4. Most of the modern 
backup tools (DD, Veeam, Rubrik, Commvault) support deduplication. LZ4 
algorithm uses less CPU than zip, gzip or bzip2 and works much better 
with deduplication algorithms employed by the backup tools. This is 
actually a very big and positive change.

Disclosure:

I used to work for Commvault as a senior PS engineer. Commvault was the 
first tool on the market to combine LZ4 and deduplication.

Regards


-- 
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
https://dbwhisperer.wordpress.com




pgsql-general by date:

Previous
From: "Nick Renders"
Date:
Subject: Re: PostgreSQL 14: pg_dump / pg_restore error: could not write to the communication channel: Broken pipe
Next
From: Tom Lane
Date:
Subject: Re: PostgreSQL 14: pg_dump / pg_restore error: could not write to the communication channel: Broken pipe