[ADMIN] pg_backrest no longer backing up - Mailing list pgsql-admin
| From | Aldo Sarmiento |
|---|---|
| Subject | [ADMIN] pg_backrest no longer backing up |
| Date | |
| Msg-id | CAHX=r6yti58x1z4y8q_zqr_P+8_Aq5_uPKSJDu1xDaV+rtjQmA@mail.gmail.com Whole thread Raw |
| Responses |
Re: [ADMIN] pg_backrest no longer backing up
|
| List | pgsql-admin |
Hello,
Sometime in December in 2016 we ran out of space for our backup directory, which we increased promptly. However, it appears that backrest stopped taking backups since the incident.
Versions of things:
pg_backrest 0.75
postgresql 9.4
ubuntu 14.04 (This is the backup server, not master)
Couple things I noticed:
postgres@pgdedicated:/backup/backrest/log$ cat prod-9.4-backup-20170308.log
-------------------PROCESS START-------------------
2017-03-08 01:30:02.034 T00 INFO: backup start: type = diff
2017-03-08 01:30:02.835 T00 INFO: last backup label = 20161204-025729F, version = 0.75
2017-03-08 01:30:02.911 T00 INFO: executing pg_start_backup() with label "pg_backrest backup started 2017-03-08 01:30:02": backup will begin after the requested immediate checkpoint completes
2017-03-08 01:30:02.976 T00 ERROR: process terminated on signal or exception, 0 threads stopped
postgres@pgdedicated:/backup/backrest/log$ ls -lah /backup/backrest/backup/prod-9.4/
total 44K
drwxrwxr-x 11 postgres postgres 4.0K Dec 22 14:02 .
drwxrwxr-x 3 postgres postgres 4.0K Mar 15 2016 ..
drwxrwxr-x 3 postgres postgres 4.0K Nov 20 03:03 20161120-030305F
drwxrwxr-x 3 postgres postgres 4.0K Nov 27 02:53 20161127-025355F
drwxrwxr-x 3 postgres postgres 4.0K Dec 3 02:24 20161127-025355F_20161203-022437D
drwxrwxr-x 3 postgres postgres 4.0K Dec 4 02:57 20161204-025729F
drwxrwxr-x 3 postgres postgres 4.0K Dec 5 02:24 20161204-025729F_20161205-022427D
drwxrwxr-x 3 postgres postgres 4.0K Dec 6 02:25 20161204-025729F_20161206-022512D
drwxrwxr-x 3 postgres postgres 4.0K Dec 7 02:28 20161204-025729F_20161207-022814D
drwxrwxr-x 3 postgres postgres 4.0K Dec 8 02:27 20161204-025729F_20161208-022730D
drwxrwxr-x 3 postgres postgres 4.0K Dec 9 02:28 20161204-025729F_20161209-022855D
lrwxrwxrwx 1 postgres postgres 33 Dec 9 02:28 latest -> 20161204-025729F_20161209-022855D
postgres@pgdedicated:/backup/backrest/log$ cat /etc/cron.d/backrest
# Vacuum every Sunday before Full backup
#05 00 * * 0 postgres vacuumdb -qaz
# Full backup every Sunday
#30 01 * * 0 postgres /usr/lib/backrest/bin/pg_backrest.pl --stanza=prod-9.4 --type=full backup
# Differential backup all other days
#30 01 * * 1,2,3,4,5,6 postgres /usr/lib/backrest/bin/pg_backrest.pl --stanza=prod-9.4 --type=diff backup
postgres@pgdedicated:/backup/backrest/log$ cat /etc/pg_backrest.conf
[global:general]
repo-path=/backup/backrest
[global:backup]
start-fast=y
[global:expire]
retention-full=6 # This was originally 8, dialed it down to 6 when ran out of space
retention-diff=6
retention-archive-type=full
retention-archive=6 # This was originally 8, dialed it down to 6 when ran out of space
[prod-9.4]
db-path=/var/lib/postgresql/9.4/replica
Question: Not sure when the backrest cron was commented out, but how is it still attempting to do backups without this setup? I'm assuming it's being invoked some other way?
Also, any idea what the issue is with the error?
pgsql-admin by date: