Re: getting pg_basebackup to use remote destination - Mailing list pgsql-general

From Chuck Martin
Subject Re: getting pg_basebackup to use remote destination
Date
Msg-id CAFw6=U35kx1PVsxo3oZboY8YdO327FX0hctL=eaqBvshK=2o6g@mail.gmail.com
Whole thread Raw
In response to Re: getting pg_basebackup to use remote destination  (Jeff Janes <jeff.janes@gmail.com>)
Responses Re: getting pg_basebackup to use remote destination  (Stephen Frost <sfrost@snowman.net>)
List pgsql-general
On Mon, Dec 31, 2018 at 12:05 PM Jeff Janes <jeff.janes@gmail.com> wrote:
On Sun, Dec 30, 2018 at 6:17 PM Chuck Martin <clmartin@theombudsman.com> wrote:
Maybe I need to rethink ths and take Jeff's advice. I executed this:

pg_basebackup -h [main server's URL] -U postgres -P -v -X s -D /mnt/dbraid/data

8 hours ago, and it is now still at 1%. Should it be that slow? The database in question is about 750 GB, and both servers are on the same GB ethernet network.

Over gigabit ethernet, it should not be that slow.  Unless the network is saturated with other traffic or something.  Might be time to call in the network engineers.  Can you transfer static files at high speeds between those two hosts using scp or rsync?  (Or use some other technique to take PostgreSQL out of the loop and see if your network is performing as it should)

Are you seeing transfers at a constant slow rate, or are their long freezes or something?  Maybe the initial checkpoint was extremely slow?  Unfortunately -P option (even with -v) doesn't make this easy to figure out.  So alas it's back to old school stopwatch and a pen and paper (or spreadsheet).

Cheers,

Jeff
Using iperf, the transfer speed between the two servers (from the main to the standby) was 938 Mbits/sec. If I understand the units correctly, it is close to what it can be. 

Your earlier suggestion was to do the pg_basebackup locally and rsync it over. Maybe that would be faster. At this point, it is saying it is 6% through, over 24 hours after being started.

Chuck Martin
Avondale Software
 

pgsql-general by date:

Previous
From: Jeff Janes
Date:
Subject: Re: getting pg_basebackup to use remote destination
Next
From: Mark Mikulec
Date:
Subject: Can't quote_literal with COPY FROM PROGRAM