Re: My Pgbackrest Backups Take Up Too Much Space - Mailing list pgsql-admin

From rams nalabolu
Subject Re: My Pgbackrest Backups Take Up Too Much Space
Date
Msg-id CAHq7gtCHy0OrNhbhU1NhxZZqQ5CxzcF5jiNxjAeWrGnkG-H+1A@mail.gmail.com
Whole thread Raw
In response to My Pgbackrest Backups Take Up Too Much Space  (Abdullah Ergin <abdullaherginwork@gmail.com>)
List pgsql-admin
Under global parameters section you can add 

  • compress-type - determines the compression algorithm used by the backupand archive-push commands. The default is gz (Gzip) but zst (Zstandard) is recommended because it is much faster and provides compression similar to gzzst has been supported by the compress-type option since v2.27. See Compress Type for more details.

Default it will install LZ4 
And you can add compression_level=3 or based on your need.

To delete backups there is another option 
Repo-retention-diff or repo-retention-full 

Thanks

On Tue, Feb 18, 2025 at 6:17 AM Abdullah Ergin <abdullaherginwork@gmail.com> wrote:
Hi,
I have a 9.3 TB database. And when I take a full backup, the space it occupies in the repo is 8.7 TB. This compression is not enough for me. It needs to be compressed more, I did not enter any parameters on my backup server for compression, so it compresses with default parameters (gz). Normally gz compresses well, why is there so little compression?

Also my archive files, the two days archive file of my stanza is now 744GB. I understand that my old archive files are not deleted after my incr backup. Wouldn't my old archive files be redundant after my incremental backups, wouldn't it be better to delete them? What is the parameter to ensure deletion?

backup info:
        full backup: 20250215-000009F
            timestamp start/stop: 2025-02-15 00:00:09+03 / 2025-02-16 03:12:36+03
            wal start/stop: 0000000200001FC90000008B / 0000000200001FCF00000061
            database size: 9375.6GB, database backup size: 9375.6GB
            repo1: backup size: 8754.4GB

        incr backup: 20250215-000009F_20250217-020004I
            timestamp start/stop: 2025-02-17 02:00:04+03 / 2025-02-17 04:10:25+03
            wal start/stop: 000000020000207700000090 / 0000000200002077000000A5
            database size: 9392.0GB, database backup size: 897.7GB
            repo1: backup size: 712.7GB
            backup reference total: 1 full

        incr backup: 20250215-000009F_20250218-020005I
            timestamp start/stop: 2025-02-18 02:00:05+03 / 2025-02-18 02:14:59+03
            wal start/stop: 0000000200002087000000F8 / 000000020000208800000008
            database size: 9419.4GB, database backup size: 364.8GB
            repo1: backup size: 54.4GB
            backup reference total: 1 full, 1 incr


As you can see here, there is a noticeable difference only in the last incr backup.

This is how much space my archive file takes up:
744G    main_stanza

Backup config parameters:
backup host:
[global]
repo1-bundle=y
repo1-block=y
repo1-path=/mnt/dbbackup
repo1-retention-full=1
process-max=4
start-fast=y
log-level-console=info
log-level-file=info

[main_stanza]
pg1-host=master
pg1-path=/mnt/postgres/13/data
pg1-port=5432

master host:
[global]
log-level-file=info
repo1-host=backup
start-fast=y
archive-async=y
process-max=2
spool-path             = /var/spool/pgbackrest

[main_stanza]
pg1-path=/mnt/postgres/13/data
pg1-port=5432

pgsql-admin by date:

Previous
From: Scott Ribe
Date:
Subject: Re: DB slowness
Next
From: kaido vaikla
Date:
Subject: Replica and transaction