Re: Improving compressibility of WAL files - Mailing list pgsql-hackers

From Aidan Van Dyk
Subject Re: Improving compressibility of WAL files
Date
Msg-id 20090109162238.GK12094@yugib.highrise.ca
Whole thread Raw
In response to Re: Improving compressibility of WAL files  ("Kevin Grittner" <Kevin.Grittner@wicourts.gov>)
Responses Re: Improving compressibility of WAL files
Re: Improving compressibility of WAL files
List pgsql-hackers
All that is useless until we get a %l in archive_command...

*I* didn't see an easy way to get at the "written" size later on in the
chain (i.e. in the actual archiving), so I took the path of least
resitance.

The reason *I* shy way from pg_lesslog and pg_clearxlogtail, is that
they seem to possibly be frail... I'm just scared of somethign changing
in PG some time, and my pg_clearxlogtail not nowing, me forgetting to
upgrade, and me not doing enough test of my actually restoring backups...

Sure, it's all me being neglgent, but the simpler, the better...

If I wrapped this zeroing in GUC, people could choose wether to pay the
penalty or not, would that satisfy anyone?

Again, *I* think that the force_switch case is going to happen when the
admin's quite happy to pay that penalty...  But obviously not
everyone...

a.

* Kevin Grittner <Kevin.Grittner@wicourts.gov> [090109 11:01]:
> >>> Greg Smith <gsmith@gregsmith.com> wrote: 
> > I thought at one point that the direction this was going toward was
> to 
> > provide the size of the WAL file as a parameter you can use in the 
> > archive_command:  %p provides the path, %f the file name, and now %l
> the 
> > length.  That makes an example archive command something like:
> > 
> > head -c "%l" "%p" | gzip > /mnt/server/archivedir/"%f"
>  
> Hard to beat for performance.  I thought there was some technical
> snag.
>  
> > Expanding it back to always be 16MB on the other side might require
> some 
> > trivial script, can't think of a standard UNIX tool suitable for that
> but 
> > it's easy enough to write.
>  
> Untested, but it seems like something close to this would work:
>  
> cat $p $( dd if=/dev/null blocks=1 ibs=$(( (16 * 1024 * 1024) - $(stat
> -c%s $p) )) )
>  
> -Kevin
> 
> -- 
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

pgsql-hackers by date:

Previous
From: Cédric Villemain
Date:
Subject: Re: Proposal: new border setting in psql
Next
From: Tom Lane
Date:
Subject: Re: SET TRANSACTION and SQL Standard