Re: Making pg_standby compression-friendly - Mailing list pgsql-hackers

From Charles Duffy
Subject Re: Making pg_standby compression-friendly
Date
Msg-id e4ccc24e0810222354n395c1082r3aef0f8e065659dd@mail.gmail.com
Whole thread Raw
In response to Re: Making pg_standby compression-friendly  (Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>)
List pgsql-hackers
On Thu, Oct 23, 2008 at 1:15 AM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote:
Charles Duffy wrote:
I'm interested in compressing archived WAL segments in an environment
set up for PITR in the interests of reducing both network traffic and
storage requirements. However, pg_standby presently checks file sizes,
requiring that an archive segment be exactly the right size to be
considered valid. The idea of compressing log segments is not new --
the clearxlogtail project in pgfoundry provides a tool to make such
compression more effective, and is explicitly intended for said
purpose -- but as of 8.3.4, pg_standby appears not to support such
environments; I propose adding such support.

Can't you decompress the files in whatever script you use to copy them to the archive location?

To be sure I understand -- you're proposing a scenario in which the archive_command on the master compresses the files, passes them over to the slave while compressed, and then decompresses them on the slave for storage in their decompressed state? That succeeds in the goal of decreasing network bandwidth, but (1) isn't necessarily easy to implement over NFS, and (2) doesn't succeed in decreasing storage requirements on the slave.

(While pg_standby's behavior is to delete segments which are no longer needed to keep a warm standby slave running, I maintain a separate archive for PITR use with hardlinked copies of those same archive segments; storage on the slave is a much bigger issue in this environment than it would be if the space used for segments were being deallocated as soon as pg_standby chose to unlink them).


[Heikki, please accept my apologies for the initial off-list response; I wasn't paying enough attention to gmail's default reply behavior].

pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: PGDay.it collation discussion notes
Next
From: Greg Stark
Date:
Subject: Re: psql Feature request \set query