Thread: BUG #18179: Cluster History Error

BUG #18179: Cluster History Error

From
PG Bug reporting form
Date:
The following bug has been logged on the website:

Bug reference:      18179
Logged by:          Abdullah Ergin
Email address:      abdullaherginwork@gmail.com
PostgreSQL version: 13.0
Operating system:   Ubuntu
Description:

Hi Dear,

I'm getting an error on my replica cluster. I could not find the cause or
solution of this error. 
Error detail:could not restore file "00000004.history" from archive: command
not found

Best Regard


Re: BUG #18179: Cluster History Error

From
Michael Paquier
Date:
On Thu, Nov 02, 2023 at 09:59:22AM +0000, PG Bug reporting form wrote:
> I'm getting an error on my replica cluster. I could not find the cause or
> solution of this error.
> Error detail:could not restore file "00000004.history" from archive: command
> not found

You will need to provide more evidence if you think that this is a bug
in the core code of Postgres.  From what you are reporting, this is
just saying that a restore_command has failed before it could not find
the command to use.  This depends on your configuration and the
environment setup where Postgres is running.
--
Michael

Attachment

Re: BUG #18179: Cluster History Error

From
Abdullah Ergin
Date:
The log of the error I encountered is as follows:

```
● postgresql@13-mydb.service - PostgreSQL Cluster 13-mydb
   Loaded: loaded (/lib/systemd/system/postgresql@.service; enabled-runtime; vendor preset: enabled)
   Active: failed (Result: protocol) since Thu 2023-11-09 17:01:44 +03; 1min 32s ago
  Process: 767207 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 13-mydb start (code=exited, status=1/FAILURE)
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: sh: 1: pgbackrest: not found
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.700 +03 [767228] FATAL: could not restore file "00000004.history" from archive: command not found
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: startup process (PID 767228) exited with exit code 1
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: aborting startup due to startup process failure
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.745 +03 [767227] LOG: database system is shut down
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: pg_ctl: could not start server
 Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: Examine the log output.
 Nov 09 17:01:44 se-prd-db02 systemd[1]: postgresql@13-mydb.service: Can't open PID file /run/postgresql/13-mydb.pid (yet?) after start: Operation not permitted
 Nov 09 17:01:44 se-prd-db02 systemd[1]: postgresql@13-mydb.service: Failed with result 'protocol'.
 Nov 09 17:01:44 se-prd-db02 systemd[1]: Failed to start PostgreSQL Cluster 13-mydb.
```

Based on this, I checked the `postgresql.auto.conf` on my master server, and the restore information found there is as follows:

```
# Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
restore_command = 'pgbackrest --stanza=mydb_stanza archive-get %f "%p'
```

The archive command information in `postgresql.conf` is as follows:

```
# - Archiving -
archive_mode = on
archive_command = 'pgbackrest --stanza=mydb_stanza archive-push %p'
```
Thank you.

Re: BUG #18179: Cluster History Error

From
Laurenz Albe
Date:
On Mon, 2023-11-13 at 21:00 +0300, Abdullah Ergin wrote:
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: sh: 1: pgbackrest: not found
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.700 +03 [767228] FATAL: could not
restorefile "00000004.history" from archive: command not found 
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: startup process
(PID767228) exited with exit code 1 
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: aborting startup
dueto startup process failure 
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.745 +03 [767227] LOG: database system is
shutdown 
>
> Based on this, I checked the `postgresql.auto.conf` on my master server, and the restore information found there is
asfollows: 
>
> # Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
> restore_command = 'pgbackrest --stanza=mydb_stanza archive-get %f "%p'

Use the absolute path for "pgbackrest".
Did you install pgBackRest on the standby?

Yours,
Laurenz Albe



Re: BUG #18179: Cluster History Error

From
Abdullah Ergin
Date:
Elbette, işte İngilizce çevirisi:

---

I'm relatively new to database expertise, are you referring to the absolute path as follows:

postgresql.auto.conf;
#Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
restore_command = '/usr/bin/pgbackrest --stanza=mydb_stanza archive-get %f "%p"'

postgresql.conf;
# - Archiving -
archive_mode = on
archive_command = '/usr/bin/pgbackrest --stanza=mydb_stanza archive-push %p'

Will this operation affect my other clusters since I will perform it on the master server?
Thanks

Laurenz Albe <laurenz.albe@cybertec.at>, 13 Kas 2023 Pzt, 22:05 tarihinde şunu yazdı:
On Mon, 2023-11-13 at 21:00 +0300, Abdullah Ergin wrote:
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: sh: 1: pgbackrest: not found
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.700 +03 [767228] FATAL: could not restore file "00000004.history" from archive: command not found
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: startup process (PID 767228) exited with exit code 1
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.703 +03 [767227] LOG: aborting startup due to startup process failure
>  Nov 09 17:01:44 se-prd-db02 postgresql@13-mydb[767207]: 2023-11-09 17:01:44.745 +03 [767227] LOG: database system is shut down
>
> Based on this, I checked the `postgresql.auto.conf` on my master server, and the restore information found there is as follows:
>
> # Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
> restore_command = 'pgbackrest --stanza=mydb_stanza archive-get %f "%p'

Use the absolute path for "pgbackrest".
Did you install pgBackRest on the standby?

Yours,
Laurenz Albe

Re: BUG #18179: Cluster History Error

From
Laurenz Albe
Date:
On Mon, 2023-11-13 at 22:42 +0300, Abdullah Ergin wrote:
> I'm relatively new to database expertise, are you referring to the absolute path as follows:
>
> postgresql.auto.conf;
> #Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
> restore_command = '/usr/bin/pgbackrest --stanza=mydb_stanza archive-get %f "%p"'

Yes, precisely.  But if te executable is in /usr/bin, it should be found
anyway... Is there really a /usr/bin/pgbackrest on the standby server?

Yours,
Laurenz Albe



Re: BUG #18179: Cluster History Error

From
Abdullah Ergin
Date:
pgBackRest is only installed on my master and backup servers. The directory where it is installed is '/usr/bin/pgbackrest.'

However, pgBackRest is not installed on my replica server.

Laurenz Albe <laurenz.albe@cybertec.at>, 13 Kas 2023 Pzt, 23:13 tarihinde şunu yazdı:
On Mon, 2023-11-13 at 22:42 +0300, Abdullah Ergin wrote:
> I'm relatively new to database expertise, are you referring to the absolute path as follows:
>
> postgresql.auto.conf;
> #Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
> restore_command = '/usr/bin/pgbackrest --stanza=mydb_stanza archive-get %f "%p"'

Yes, precisely.  But if te executable is in /usr/bin, it should be found
anyway... Is there really a /usr/bin/pgbackrest on the standby server?

Yours,
Laurenz Albe

Re: BUG #18179: Cluster History Error

From
Laurenz Albe
Date:
On Tue, 2023-11-14 at 00:04 +0300, Abdullah Ergin wrote:
> Laurenz Albe <laurenz.albe@cybertec.at>, 13 Kas 2023 Pzt, 23:13 tarihinde şunu yazdı:
> > On Mon, 2023-11-13 at 22:42 +0300, Abdullah Ergin wrote:
> > > I'm relatively new to database expertise, are you referring to the absolute path as follows:
> > >
> > > postgresql.auto.conf;
> > > #Recovery settings generated by pgBackRest restore on 2021-09-29 01:04:40
> > > restore_command = '/usr/bin/pgbackrest --stanza=mydb_stanza archive-get %f "%p"'
> >
> > Yes, precisely.  But if te executable is in /usr/bin, it should be found
> > anyway... Is there really a /usr/bin/pgbackrest on the standby server?
>
> pgBackRest is only installed on my master and backup servers. The directory where
> it is installed is '/usr/bin/pgbackrest.'
>
> However, pgBackRest is not installed on my replica server.

Then you shouldn't use it in your "restore_command".  Best comment that out.

This is no PostgreSQL bug.

Yours,
Laurenz Albe