Re: Improving compressibility of WAL files - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Improving compressibility of WAL files
Date
Msg-id 1231518667.18005.475.camel@ebony.2ndQuadrant
Whole thread Raw
In response to Re: Improving compressibility of WAL files  (Bruce Momjian <bruce@momjian.us>)
Responses Re: Improving compressibility of WAL files  (Aidan Van Dyk <aidan@highrise.ca>)
Re: Improving compressibility of WAL files  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Fri, 2009-01-09 at 09:31 -0500, Bruce Momjian wrote:
> Tom Lane wrote:
> > Bruce Momjian <bruce@momjian.us> writes:
> > > Tom Lane wrote:
> > >> Isn't this redundant given the existence of pglesslog?
> > 
> > > It does the same as pglesslog, but is simpler to use because it is
> > > automatic.
> > 
> > Which also means that everyone pays the performance penalty whether
> > they get any benefit or not.  The point of the external solution
> > is to do the work only in installations that get some benefit.
> > We've been over this ground before...
> 
> If there is a performance penalty, you are right, but if the zeroing is
> done as part of the archiving, it seems near zero cost enough to do it
> all the time, no?

It can already be done as part of the archiving, using an external tool
as Tom notes.

Yes, we could make the archiver do this, but I see no big advantage over
having it done externally. It's not faster, safer, easier. Not easier
because we would want a parameter to turn it off when not wanted.

The patch as stands is IMHO not acceptable because the work to zero the
file is performed by the unlucky backend that hits EOF on the current
WAL file, which is bad enough, but it is also performed while holding
WALWriteLock. 

I like Greg Smith's analysis of this and his conclusion that we could
provide a %l option, but even that would require work to have that info
passed to the archiver. Perhaps inside the notification file which is
already written and read by the write processes. But whether that can or
should be done for this release is a different debate.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: foreign_data test fails with non-C locale
Next
From: Heikki Linnakangas
Date:
Subject: Hot standby, RestoreBkpBlocks and cleanup locks