Thread: Parallel pg_basebackup
Hi guys,
I was wondering if there is a proposal for parallelize pg_basebackup (sorry if there is already one, I looked into the inbox and did not find any).
I made a simple test and the performance gain is pretty considerable using parallel rsync processes vs. single pg_basebackup.
i.e., for 46GB of data I had the following results:
rsync, parallel, including the xlogs
real 18m52.674s
user 33m54.728s
sys 13m44.604s
(-x option)
real 23m26.802s
user 0m24.392s
sys 3m55.116s
Actually, AWS services limits the bandwidth per process but if you have parallel processes, none of them will have the IOPS decreased which will be translated in a great improvement. Also the network bandwidth is strong between the regions.
Thanks guys!
--
Emanuel Calvo
On Fri, 2013-08-23 at 10:51 -0300, Emanuel Calvo wrote: > I was wondering if there is a proposal for parallelize pg_basebackup There isn't one, but after some talk behind the scenes, I think we should definitely look into it.
That looks promising! Thanks Peter!
2013/8/24 Peter Eisentraut <peter_e@gmx.net>
On Fri, 2013-08-23 at 10:51 -0300, Emanuel Calvo wrote:There isn't one, but after some talk behind the scenes, I think we
> I was wondering if there is a proposal for parallelize pg_basebackup
should definitely look into it.
--
Emanuel Calvo