Re: rsync and streaming replication - Mailing list pgsql-admin

From Scott Ribe
Subject Re: rsync and streaming replication
Date
Msg-id 48B4CE29-7776-4D7A-8AA5-D45A4F77BB75@elevated-dev.com
Whole thread Raw
In response to Re: rsync and streaming replication  (Jean-Armel Luce <jaluce06@gmail.com>)
Responses Re: rsync and streaming replication  (Cédric Villemain <cedric.villemain.debian@gmail.com>)
List pgsql-admin
On Nov 16, 2011, at 10:11 AM, Jean-Armel Luce wrote:

> You are right. I used -a, and I was wanting to be more meaningful so I wrote --all in my post.
> Please read --archive insterad of --all

Oh, OK. Still seems odd that it took so much longer. Granted, for the files with different timestamps but identical
contents,it then syncs them. But it does so by checksumming blocks, comparing checksums, and sending only blocks that
aredifferent over the network. Granted, it has to send some checksums over the network, but that's pretty minor
traffic.I believe you said you'd seen 125Mb/s over the network? Is that actually accurate? Does the network connection
havehigh latency? 

Also, I believe you said -z seemed to slow it down? That has not been my experience at all with rsync'ing pg databases.
Betweenall the values that are stored as plain text, and the redundancies in indexes, I usually see a good speed
increasefrom compressing the data in transit. 

I'm certainly glad that you've got a 3x speed increase--that's significant progress. But still, something seems odd
aboutthe performance you've reported, so I'm left wondering about what could cause that. Disk performance glitch at one
endor the other, network performance, CPU load??? 

One thing worth doing I think is to use --stats on every test, so you can see every time how many files and how much
datais actually transferred. Also, if you're sitting there watching, sometimes --progress can be informative... 

--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice





pgsql-admin by date:

Previous
From: Jean-Armel Luce
Date:
Subject: Re: rsync and streaming replication
Next
From: Cédric Villemain
Date:
Subject: Re: rsync and streaming replication