Re: pgbackrest check info fails without success - Mailing list pgsql-general

From KK CHN
Subject Re: pgbackrest check info fails without success
Date
Msg-id CAKgGyB8UU3SzqwquEF1w_ufuFqJU7bsXdLG5zxSbcbD9T57Oaw@mail.gmail.com
Whole thread Raw
In response to Re: pgbackrest check info fails without success  (Ruben Morais <ruben.gmorais@gmail.com>)
Responses Re: pgbackrest check info fails without success
List pgsql-general


On Wed, Aug 13, 2025 at 2:49 PM Ruben Morais <ruben.gmorais@gmail.com> wrote:
Hi, 

Simply update the postgres.conf with the desired stanza, and add the new one in the pgbackrest.conf file.
After that, run stanza-create with the new name.
Once everything is OK, go to the repository directory and remove all files related to that stanza.

With this approach, there’s no downtime in production (just make sure beforehand that, in a production environment, if you don't need the backups already done with old stanza name).


BR
Rúben Morais


Done the new stanza creation, but the issue of  stanza check fails still persists.... 

The pgbackrest  stanza check info still fails with error : 

INFO: check repo1 configuration (primary)
INFO: check repo1 archive for WAL (primary)
ERROR: [082]: WAL segment 000000010000001D000000A1 was not archived before the 60000ms timeout
       HINT: check the archive_command to ensure that all options are correct (especially --stanza).
       HINT: check the PostgreSQL server log for errors.


 seems the 60000ms  the WAL files are not archived to the remote site.  At DB Cluster side server N/W there we are experiencing latency issues. ,  so I suspect the WAL files are not archived to the remote repo server within this limit of 60s  to the remote Repo server.      Can we increase this limit to 60000ms? Is this advisable what the parameter to  add in the pgbackrest.conf in DB Server ?
[my_cluster]archive-timeout=120io-timeout=120 ??




DB server log says
 5-08-13 16:15:50.666 P00   INFO: archive-push command end: aborted with exception [062]
2025-08-13 16:15:50 IST LOG:  archive command failed with exit code 62
2025-08-13 16:15:50 IST DETAIL:  The failed archive command was: pgbackrest --stanza=K_Repo archive-push pg_wal/0000000100000018000000C8
2025-08-13 16:15:50 IST WARNING:  archiving write-ahead log file "0000000100000018000000C8" failed too many times, will try again later


INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001900000010] --archive-async --compress-type=zst --exec-id=4180767-84540d74 --log-level-console=info --log-level-file=debug --log-level-stderr=info --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --repo1-path=/DB_BKUP --spool-path=/var/spool/pgbackrest --stanza=K_Repo
ERROR: [062]: stop file exists for stanza K_Repo
INFO: archive-push command end: aborted with exception [062]
2025-08-13 16:25:51 IST LOG:  archive command failed with exit code 62
2025-08-13 16:25:51 IST DETAIL:  The failed archive command was: pgbackrest --stanza=K_Repo archive-push pg_wal/000000010000001900000010
2025-08-13 16:25:51 IST WARNING:  archiving write-ahead log file "000000010000001900000010" failed too many times, will try again later


So the N/W latency badly affects the Pgbackrest working ? 


On Wed, Aug 13, 2025 at 9:48 AM KK CHN <kkchn.in@gmail.com> wrote:





On Wed, Aug 13, 2025 at 1:24 PM Ruben Morais <ruben.gmorais@gmail.com> wrote:
Hi,

All seems ok, but in your check command you use a different stanza name from the configuration and from what you have in postgres.log file.

Created stanza  successfully :    on my Repo server
[root@db01 ~]# sudo -u pgbackrest pgbackrest --stanza=K_Repo --log-level-console=info stanza-create
successful

But  info check fails multiple times no even success for one time
[root@db01 ~]# sudo -u pgbackrest pgbackrest --stanza=Kerala_Repo --log-level-console=info check    <<< change Kerala_Repo to K_Repo 
fails always with error

Rúben Morais


I think  I messed up with Stanza names ...

I want to  delete the wrong stanza and recreate a new stanza.    Please correct me if I'm wrong, for recreating the stanza with the following steps ?

1. Stop the DB cluster on the DB server machine(This is a production server, so down time constraints )  is this step mandatory if  I wish to recreate the stanza ?
( Or can I do without stopping the DB cluster on the DB Server machine and follow the below steps )
2. sudo -u pgbackrest  pgbackers --stanza=K_Repo  stop  ( On Repo server )  and
3. sudo -u enterprisedb pgbackrest --stanza=K_Repo stop ( On the DB Server M/C also ?) 

4. sudo -u pgbackrest pgbackrest  --stanza=K_Repo  stanza-delete ( On Repo server )  do I need to repeat this on  DB Server also as ( sudo -u enterprisedb pgbackrest  --stanza=K_Repo  stanza-delete   ?)  


Then
5.    Execute fresh stanza-create step with correct stanza name .

If the above 5 sequence are correct let me know, or if any thing repetition or not required  in the steps 1 to 4 let me know.  


Krishane
 

On Wed, Aug 13, 2025 at 8:32 AM KK CHN <kkchn.in@gmail.com> wrote:
Hi all,

I have configured  pgbackrest  on ( RHEL 9.3  , pgbackrest 2.52.1  database version 16 ) for a remote repo machine (RHEL 9.3, pgbackrest 2.52.1 ) 

I have created the stanza   K_repo

archive_command = 'pgbackrest --stanza=K_Repo archive-push %p'    in postgresql.conf


Created stanza  successfully :    on my Repo server
[root@db01 ~]# sudo -u pgbackrest pgbackrest --stanza=K_Repo --log-level-console=info stanza-create
successful

But  info check fails multiple times no even success for one time
[root@db01 ~]# sudo -u pgbackrest pgbackrest --stanza=Kerala_Repo --log-level-console=info check
fails always with error

INFO: check repo1 configuration (primary)
INFO: check repo1 archive for WAL (primary)
ERROR: [082]: WAL segment 000000010000001D0000001E was not archived before the 60000ms timeout
       HINT: check the archive_command to ensure that all options are correct (especially --stanza).
       HINT: check the PostgreSQL server log for errors.
       HINT: run the 'start' command if the stanza was previously stopped.
INFO: check command end: aborted with exception [082]

I have done start, stop, start the stanza from Repo host and re-executed the  info check but failed. 

What may be the issue, what will be the fix ?  any hints most welcome..



Thank you
Krishane


here the db server log.
025-08-13 12:20:29.488 P00   INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001700000021] --archive-async --exec-id=3775881-26b1d28c --log-level-console=info --log-level-file=debug --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --spool-path=/var/spool/pgbackrest --stanza=K_Repo
2025-08-13 12:20:42 IST LOG:  checkpoint starting: time
2025-08-13 12:20:43.691 P00   INFO: pushed WAL file '000000010000001700000021' to the archive asynchronously
2025-08-13 12:20:43.691 P00   INFO: archive-push command end: completed successfully (14205ms)
2025-08-13 12:20:43.698 P00   INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001700000022] --archive-async --exec-id=3775955-a5353845 --log-level-console=info --log-level-file=debug --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --spool-path=/var/spool/pgbackrest --stanza=K_Repo
2025-08-13 12:20:57.901 P00   INFO: pushed WAL file '000000010000001700000022' to the archive asynchronously
2025-08-13 12:20:57.901 P00   INFO: archive-push command end: completed successfully (14205ms)
2025-08-13 12:20:57.908 P00   INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001700000023] --archive-async --exec-id=3776006-fc68ed25 --log-level-console=info --log-level-file=debug --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --spool-path=/var/spool/pgbackrest --stanza=K_Repo
2025-08-13 12:21:01.110 P00   INFO: pushed WAL file '000000010000001700000023' to the archive asynchronously
2025-08-13 12:21:01.110 P00   INFO: archive-push command end: completed successfully (3203ms)
2025-08-13 12:21:01.118 P00   INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001700000024] --archive-async --exec-id=3776015-5508d157 --log-level-console=info --log-level-file=debug --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --spool-path=/var/spool/pgbackrest --stanza=K_Repo
2025-08-13 12:21:15.321 P00   INFO: pushed WAL file '000000010000001700000024' to the archive asynchronously
2025-08-13 12:21:15.321 P00   INFO: archive-push command end: completed successfully (14204ms)
2025-08-13 12:21:15.328 P00   INFO: archive-push command begin 2.52.1: [pg_wal/000000010000001700000025] --archive-async --exec-id=3776065-a6d63fe2 --log-level-console=info --log-level-file=debug --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=6 --repo1-host=10.50.2.202 --repo1-host-user=pgbackrest --spool-path=/var/spool/pgbackrest --stanza=K_Repo








pgsql-general by date:

Previous
From: Ruben Morais
Date:
Subject: Re: pgbackrest check info fails without success
Next
From: Rachel Roch
Date:
Subject: Re: Bitnami deprecation