Hi Steven:
On Fri, Mar 13, 2015 at 9:03 PM, Steven Lembark <lembark@wrkhors.com> wrote:
> > The thing is you can use desktop class machines for the slave. If you do
.....
> If load on the backup server becomes an issue you might be able to
> make incremental pg_dump's onto tmpfs.
I'm curious, how do you make incremental backups using pg_dump? I do
not know it, haven't been able to find it in the docs and it would be
a great solutions for a couple of problems I have.
> Advantage there is that the
> dump iteslf has effectively no write I/O overhead: you can dump to
> tmpfs and then [bg]zip to stable storage w/o beating up the disks,
> which becomes a real problem with comodity-grade hardware.
I do not think this ramdisks are a good thing for that. Let me
explain, if you have tmpfs, you have devent pipe support in the shell.
If you can put the uncompressed dump in a ramdisk, you are going to be
able to fit the compressed one in a write-through disk cache. So you
just dump compressed ( either piping through a compressor or using
pg_dump's custom compressed format ) and let the write cache do its
magic ( also, it's going to beat the disk less ( or equal, if your RAM
is really big for the task ). If you use a ramdisk and compress you
have less ram to cache the compressed dump, write cache is gonna
perform worse, disk is gonna be beaten more ). Also, tmpfs means less
ram for disk cache and shared buffer for the database. On a machine
needing the RAM my money is in it making the things slower ( on an
scenario like 8Gb db on 64Gb machine whatever you do is going to be
fast enough, so just peek the simpler to code ).
And regarding compression, pg_dump's one does normally perform good
enough unless you have strange data, I've done several tests and found
in normal data it does not pay to either raise the level ( little
space gain for the cycles taken ) or try alternate ones ( I normally
always use custom format due to it's ability to easily do partial
restores. Compressing it outside means you need to decompress to use.
). Also, I did some compressor tests for intermediate files, and found
for many reading/writing using gzip/lzo was faster than uncompressed (
specially with modern multicore machines ), xz was the compression
king and I was severely disapointed by bzip2 ( for any option combo in
bzip2 I had another in xz which beat it in BOTH size AND time, YMMV ).
Regards.
Francisco Olarte.
