Thread: PgBackRest Full backup and N/W reliability
List,
[root@dbtest pgbackrest]# sudo -u postgres pgbackrest --stanza=Repo --log-level-console=info backup
2024-08-29 10:55:27.729 P00 INFO: backup command begin 2.52.1: --delta --exec-id=523103-56943986 --log-level-console=info --log-level-file=debug --pg1-host=10.15.0.202 --pg1-host-user=enterprisedb --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5 --repo1-block --repo1-bundle --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS --repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
WARN: no prior backup exists, incr backup has been changed to full
2024-08-29 10:55:30.589 P00 INFO: execute non-exclusive backup start: backup begins after the requested immediate checkpoint completes
2024-08-29 10:55:31.543 P00 INFO: backup start archive = 00000001000000850000004C, lsn = 85/4C0007F8
2024-08-29 10:55:31.543 P00 INFO: check archive for prior segment 00000001000000850000004B
I am doing a full backup using PgBackRest from a production server to Reposerver.
My connection is IPSec VPN over ILL ( 8 Mbps link) between the Production DB Server and the remote RepoServer.
I understood the bottleneck of 8 Mbps between servers. (Server NICs 10Gbps and switch)
Query : I have started the backup command and it is running (may go for hours and days as link speed is minimal) .
If the link disconnected or Network error happens before completion of the backup command Definitely the option is to reissue the backup command again.
If so, does the backup process start again from scratch ? or it resumes from where the backup process is stopped ?
If it starts from scratch I am afraid that I can''t complete the initial full backup never :(
Or is there a work around if the network connectivity is lost in between ?
Any suggestions much appreciated
Thank you ,
Krishane
[root@dbtest pgbackrest]# sudo -u postgres pgbackrest --stanza=Repo --log-level-console=info backup
2024-08-29 10:55:27.729 P00 INFO: backup command begin 2.52.1: --delta --exec-id=523103-56943986 --log-level-console=info --log-level-file=debug --pg1-host=10.15.0.202 --pg1-host-user=enterprisedb --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5 --repo1-block --repo1-bundle --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS --repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
WARN: no prior backup exists, incr backup has been changed to full
2024-08-29 10:55:30.589 P00 INFO: execute non-exclusive backup start: backup begins after the requested immediate checkpoint completes
2024-08-29 10:55:31.543 P00 INFO: backup start archive = 00000001000000850000004C, lsn = 85/4C0007F8
2024-08-29 10:55:31.543 P00 INFO: check archive for prior segment 00000001000000850000004B
ON Repo Server:
[root@dbtest backup]# date
Thursday 29 August 2024 10:58:08 AM IST
[root@dbtest backup]# du -h
Thursday 29 August 2024 10:58:08 AM IST
[root@dbtest backup]# du -h
165M ./Repo
165M
165M
[root@dbtest backup]# date
Thursday 29 August 2024 11:37:03 AM IST
[root@dbtest backup]# du -h
Thursday 29 August 2024 11:37:03 AM IST
[root@dbtest backup]# du -h
1.9G ./Repo
1.9G
ON Production Server /data/edb/as16/data directory size is 500 GB
On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:
I am doing a full backup using PgBackRest from a production server to Reposerver.
...
If so, does the backup process start again from scratch ? or it resumes from where the backup process is stopped ?
It resumes. You will see a message like this:
WARN: resumable backup 20240829-091727F of same type exists -- invalid files will be removed then the backup will resume
Any suggestions much appreciated
Boost your process-max as high as you are able to speed up your backup time.
Cheers,
Greg
On Thu, Aug 29, 2024 at 6:54 PM Greg Sabino Mullane <htamfids@gmail.com> wrote:
On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:I am doing a full backup using PgBackRest from a production server to Reposerver....If so, does the backup process start again from scratch ? or it resumes from where the backup process is stopped ?It resumes. You will see a message like this:WARN: resumable backup 20240829-091727F of same type exists -- invalid files will be removed then the backup will resumeAny suggestions much appreciatedBoost your process-max as high as you are able to speed up your backup time.
what will be the ideal process-max number to use ?
Once I update the process-max param in pgbackrest.conf , do I need to stop and start the pgback rest ?
Or just editing the pgbackrest.conf on the fly will reflect the increased process-max numbers advantage ?
Is there a limit for proces max. setting ?
Cheers,Greg
On Thu, Aug 29, 2024 at 12:08 PM KK CHN <kkchn.in@gmail.com> wrote:
On Thu, Aug 29, 2024 at 6:54 PM Greg Sabino Mullane <htamfids@gmail.com> wrote:On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:I am doing a full backup using PgBackRest from a production server to Reposerver....If so, does the backup process start again from scratch ? or it resumes from where the backup process is stopped ?It resumes. You will see a message like this:WARN: resumable backup 20240829-091727F of same type exists -- invalid files will be removed then the backup will resumeAny suggestions much appreciatedBoost your process-max as high as you are able to speed up your backup time.what will be the ideal process-max number to use ?
We can't answer that. It depends on what else your PG and PgBackRest servers are doing when "pgbackrest backup" runs.
Once I update the process-max param in pgbackrest.conf , do I need to stop and start the pgback rest ?
Or just editing the pgbackrest.conf on the fly will reflect the increased process-max numbers advantage ?
"pgbackrest backup" reads the config file when it starts.
Is there a limit for proces max. setting ?
I think you need managed DBA services.
Death to America, and butter sauce.
Iraq lobster!