Re: PgBackRest fails due to filesystem full - Mailing list pgsql-general

From KK CHN
Subject Re: PgBackRest fails due to filesystem full
Date
Msg-id CAKgGyB-M7=cVS_4bf_f=LKgZKgZMPBSr_caJ+usNQnxXEXvCag@mail.gmail.com
Whole thread Raw
In response to Re: PgBackRest fails due to filesystem full  (Greg Sabino Mullane <htamfids@gmail.com>)
Responses Re: PgBackRest fails due to filesystem full
List pgsql-general


On Tue, Apr 8, 2025 at 10:28 PM Greg Sabino Mullane <htamfids@gmail.com> wrote:
On Mon, Apr 7, 2025 at 5:32 AM KK CHN <kkchn.in@gmail.com> wrote:
ERROR: [082]: WAL segment 00000001000001EB0000004B was not archived before the 60000ms timeout

This is the part you need to focus on. Look at your Postgres logs and find out why the archiver is failing. You can also test this without trying a whole backup by using the "check" command: https://pgbackrest.org/command.html#command-check

I have run the check and it says successful !!

[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=DBCluster1_Repo  --log-level-console=info check 

[root@dbtest ~]# 2025-04-09 10:52:26.148 P00   INFO: check command begin 2.52.1: --exec-id=384808-715e8496 --log-level-console=info --log-level-file=debug --pg1-host=10.x.x.x   --pg1-host-user=enterprisedb --pg1-path=/data/edb/as16/data --pg-version-force=16 --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS --stanza=DBCluster1_Repo
2025-04-09 10:52:30.502 P00   INFO: check repo1 configuration (primary)
2025-04-09 10:52:31.003 P00   INFO: check repo1 archive for WAL (primary)
2025-04-09 10:52:36.305 P00   INFO: WAL segment 00000001000001ED00000017 successfully archived to '/data/DB_BKUPS/archive/DBCluster1_Repo/16-1/00000001000001ED/00000001000001ED00000017-8609407e8b9a1827a9d9b3e170dcc53e7af46bac.gz' on repo1
2025-04-09 10:52:36.721 P00   INFO: check command end: completed successfully (10575ms)




Then I ran 
[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=DBCluster1_Repo --type=diff backup     to test pgbackrest works fine !!!!

It says 

2025-04-09 10:53:52.521 P00   INFO: backup '20250407-150858F' cannot be resumed: resume only valid for full backup
^C2025-04-09 10:54:03.351 P00   INFO: backup command end: terminated on signal [SIGINT]

But the  # sudo -u postgres pgbackrest --stanza=DBCluster1_Repo info       command never shows such a backup   20250407-150858F exists.   The existing backups were 20250316-232631F and prior 2 full backups to this . 

Similarly   diff backups  I have the last one 20250316-232631F_20250329-172215D   and prior diffs only nothing later than this date .  and one INCR      incr backup: 20250316-232631F_20250330-083923I   noting later date than this..  So since 2025 03 30  all backups   Full/diff/incr fails  ( since the / partition ran out of space )

Nothing else reported by the info  command..  


How can I proceed to bring pgbackrest back to  take backups to normal ?     [  WAL files are missing then can we never take the Full backups / diff /inc  ? What is the workaround / solution to deal with this situation ?]

Any hints much appreciated .. 

Krishane
 
 
Cheers,
Greg

--
Enterprise Postgres Software Products & Tech Support

pgsql-general by date:

Previous
From: Brent Wood
Date:
Subject: Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm
Next
From: Achilleas Mantzios - cloud
Date:
Subject: Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm