Thread: Backup_Long Running

Backup_Long Running

From
jaya kumar
Date:

Hi Team,

 

Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.

 

DB size: 793 GB

 

We are taking pg_basebackup backup.

 

Backup File size=613G

 

Backup Running Duration: 8 Hours


--
Thanks & Regards,
Jayakumar.S
+91-9840864439.

Re: Backup_Long Running

From
Vijaykumar Jain
Date:

On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:

Hi Team,

 

Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.

 

DB size: 793 GB

 

We are taking pg_basebackup backup.


do you see network saturation, io saturation ?
generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.
where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?
basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.

now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.

so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.
there are other tools too, I used only these two.


Re: Backup_Long Running

From
jaya kumar
Date:
Thanks for your update. Can you have any link or document to configure L0 & L1 backup using pgbackrest tool. Also share the pgbackrest installation method.    

On Wed, Apr 24, 2024 at 2:50 PM Vijaykumar Jain <vijaykumarjain.github@gmail.com> wrote:

On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:

Hi Team,

 

Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.

 

DB size: 793 GB

 

We are taking pg_basebackup backup.


do you see network saturation, io saturation ?
generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.
where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?
basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.

now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.

so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.
there are other tools too, I used only these two.




--
Thanks & Regards,
Jayakumar.S
+91-9840864439.

Re: Backup_Long Running

From
Ron Johnson
Date:
PgBackRest is in the PGDG repositories (RHEL & Debian).

The documentation is thorough, and discoverable via Google,

On Wed, Apr 24, 2024 at 8:30 AM jaya kumar <kumardba27@gmail.com> wrote:
Thanks for your update. Can you have any link or document to configure L0 & L1 backup using pgbackrest tool. Also share the pgbackrest installation method.    

On Wed, Apr 24, 2024 at 2:50 PM Vijaykumar Jain <vijaykumarjain.github@gmail.com> wrote:

On Wed, Apr 24, 2024, 12:33 PM jaya kumar <kumardba27@gmail.com> wrote:

Hi Team,

 

Production database Backup is running very long hours. Any option to reduce backup time? Kindly advise me.

 

DB size: 793 GB

 

We are taking pg_basebackup backup.


do you see network saturation, io saturation ?
generally faster hardware i.e striped and or nvme disks along with a robust network link and capacity should help get the backup done quickly.
where are you taking the backup from? is the server busy doing other work or it is a dedicated machine for backups ?
basically monitor for resource saturation, if all looks good, we could take basebackup of a 10tb db in 8 hours, and in another case on a slow remote storage, backup of 2tb took 1 day.

now, pgbackrest can speedup backup processes by spawning more workers for archiving and stuff. we have taken backup on nvme disks striped of 28tb in 3 hours, bare metals servers with powerful cpu.

so , it's hardware .... else switch to pgbackrest which can take incremental/differential/full backups.
there are other tools too, I used only these two.




--
Thanks & Regards,
Jayakumar.S
+91-9840864439.

Re: Backup_Long Running

From
Adrian Klaver
Date:
On 4/24/24 00:03, jaya kumar wrote:
> Hi Team,
> 
> Production database Backup is running very long hours. Any option to 
> reduce backup time? Kindly advise me.

Hardware specifications?

Network specifications?

The actual pg_basebackup command used?

Server(s) user load?

> 
> DB size: 793 GB
> 
> We are taking pg_basebackup backup.
> 
> Backup File size=613G
> 
> Backup Running Duration: 8 Hours
> 
> 
> -- 
> Thanks & Regards,
> Jayakumar.S
> +91-9840864439.

-- 
Adrian Klaver
adrian.klaver@aklaver.com