Re: Compression of full-page-writes - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: Compression of full-page-writes
Date
Msg-id 20150102165242.GC22217@momjian.us
Whole thread Raw
In response to Re: Compression of full-page-writes  ("ktm@rice.edu" <ktm@rice.edu>)
Responses Re: Compression of full-page-writes
Re: Compression of full-page-writes
Re: Compression of full-page-writes
Re: Compression of full-page-writes
List pgsql-hackers
On Fri, Jan  2, 2015 at 10:15:57AM -0600, ktm@rice.edu wrote:
> On Fri, Jan 02, 2015 at 01:01:06PM +0100, Andres Freund wrote:
> > On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote:
> > > I still don't understand the value of adding WAL compression, given the
> > > high CPU usage and minimal performance improvement.  The only big
> > > advantage is WAL storage, but again, why not just compress the WAL file
> > > when archiving.
> > 
> > before: pg_xlog is 800GB
> > after: pg_xlog is 600GB.
> > 
> > I'm damned sure that many people would be happy with that, even if the
> > *per backend* overhead is a bit higher. And no, compression of archives
> > when archiving helps *zap* with that (streaming, wal_keep_segments,
> > checkpoint_timeout). As discussed before.
> > 
> > Greetings,
> > 
> > Andres Freund
> > 
> 
> +1
> 
> On an I/O constrained system assuming 50:50 table:WAL I/O, in the case
> above you can process 100GB of transaction data at the cost of a bit
> more CPU.

OK, so given your stats, the feature give a 12.5% reduction in I/O.  If
that is significant, shouldn't we see a performance improvement?  If we
don't see a performance improvement, is I/O reduction worthwhile?  Is it
valuable in that it gives non-database applications more I/O to use?  Is
that all?

I suggest we at least document that this feature as mostly useful for
I/O reduction, and maybe say CPU usage and performance might be
negatively impacted.

OK, here is the email I remember from Fujii Masao this same thread that
showed a performance improvement for WAL compression:
http://www.postgresql.org/message-id/CAHGQGwGqG8e9YN0fNCUZqTTT=hNr7Ly516kfT5ffqf4pp1qnHg@mail.gmail.com

Why are we not seeing the 33% compression and 15% performance
improvement he saw?  What am I missing here?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



pgsql-hackers by date:

Previous
From: "ktm@rice.edu"
Date:
Subject: Re: Compression of full-page-writes
Next
From: Andres Freund
Date:
Subject: Re: Compression of full-page-writes