Re: [REVIEW] Re: Compression of full-page-writes - Mailing list pgsql-hackers

On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote:
> I agree that there's no reason to fix an algorithm to it, unless maybe it's
> pglz. There's some initial talk about implementing pluggable compression
> algorithms for TOAST and I guess the same must be taken into consideration
> for the WAL.
> 
> --
> Arthur Silva
> 
> 
> On Thu, Sep 11, 2014 at 2:46 AM, Rahila Syed <rahilasyed.90@gmail.com>
> wrote:
> 
> > >I will repeat the above tests with high load on CPU and using the
> > benchmark
> > given by Fujii-san and post the results.
> >
> > Average % of CPU usage at user level for each of the compression algorithm
> > are as follows.
> >
> > Compression        Multiple            Single
> >
> > Off                        81.1338            81.1267
> > LZ4                      81.0998            81.1695
> > Snappy:                80.9741             80.9703
> > Pglz :                    81.2353            81.2753
> >
> > <
> > http://postgresql.1045698.n5.nabble.com/file/n5818552/CPU_utilization_user_single.png
> > >
> > <
> > http://postgresql.1045698.n5.nabble.com/file/n5818552/CPU_utilization_user.png
> > >
> >
> > The numbers show CPU utilization of Snappy is the least. The CPU
> > utilization
> > in increasing order is
> > pglz > No compression > LZ4 > Snappy
> >
> > The variance of average CPU utilization numbers is very low. However ,
> > snappy seems to be best when it comes to lesser utilization of CPU.
> >
> > As per the measurement results posted till date
> >
> > LZ4 outperforms snappy and pglz in terms of compression ratio and
> > performance. However , CPU utilization numbers show snappy utilizes least
> > amount of CPU . Difference is not much though.
> >
> > As there has been no consensus yet about which compression algorithm to
> > adopt, is it better to make this decision independent of the FPW
> > compression
> > patch as suggested earlier in this thread?. FPW compression can be done
> > using built in compression pglz as it shows considerable performance over
> > uncompressed WAL and good compression ratio
> > Also, the patch to compress multiple blocks at once gives better
> > compression
> > as compared to single block. ISTM that performance overhead introduced by
> > multiple blocks compression is slightly higher than single block
> > compression
> > which can be tested again after modifying the patch to use pglz .  Hence,
> > this patch can be built using multiple blocks compression.
> >
> > Thoughts?
> >

Hi,

The big (huge) win for lz4 (not the HC variant) is the enormous compression
and decompression speed. It compresses quite a bit faster (33%) than snappy
and decompresses twice as fast as snappy.

Regards,
Ken