Thread: Backup_Long Running
Hi Team,
Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.
DB size: 793 GB
We are taking pg_basebackup backup.
Backup File size=613G
Backup Running Duration: 8 Hours
Thanks & Regards,
Jayakumar.S+91-9840864439.
On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:
Hi Team,
Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.
DB size: 793 GB
We are taking pg_basebackup backup.
do you see network saturation, io saturation ?
generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.
where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?
basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.
now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.
so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.
there are other tools too, I used only these two.
Thanks for your update. Can you have any link or document to configure L0 & L1 backup using pgbackrest tool. Also share the pgbackrest installation method.
On Wed, Apr 24, 2024 at 2:50 PM Vijaykumar Jain <vijaykumarjain.github@gmail.com> wrote:
On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:Hi Team,
Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.
DB size: 793 GB
We are taking pg_basebackup backup.
do you see network saturation, io saturation ?generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.there are other tools too, I used only these two.
Thanks & Regards,
Jayakumar.S+91-9840864439.
PgBackRest is in the PGDG repositories (RHEL & Debian).
The documentation is thorough, and discoverable via Google,
On Wed, Apr 24, 2024 at 8:30 AM jaya kumar <kumardba27@gmail.com> wrote:
Thanks for your update. Can you have any link or document to configure L0 & L1 backup using pgbackrest tool. Also share the pgbackrest installation method.On Wed, Apr 24, 2024 at 2:50 PM Vijaykumar Jain <vijaykumarjain.github@gmail.com> wrote:On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:Hi Team,
Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.
DB size: 793 GB
We are taking pg_basebackup backup.
do you see network saturation, io saturation ?generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.there are other tools too, I used only these two.
--Thanks & Regards,Jayakumar.S
+91-9840864439.
On 4/24/24 00:03, jaya kumar wrote: > Hi Team, > > Production database Backup is running very long hours. Any option to > reduce backup time? Kindly advise me. Hardware specifications? Network specifications? The actual pg_basebackup command used? Server(s) user load? > > DB size: 793 GB > > We are taking pg_basebackup backup. > > Backup File size=613G > > Backup Running Duration: 8 Hours > > > -- > Thanks & Regards, > Jayakumar.S > +91-9840864439. -- Adrian Klaver adrian.klaver@aklaver.com