PG version 16.14 upgrade went recovery state - Mailing list pgsql-admin
| From | K P Manoj |
|---|---|
| Subject | PG version 16.14 upgrade went recovery state |
| Date | |
| Msg-id | CAMVgnQ48N4Vspjg-m4TgYEK2-O_F5L0gGEPtjVSMqQrUQ+24Dg@mail.gmail.com Whole thread |
| Responses |
Re: PG version 16.14 upgrade went recovery state
|
| List | pgsql-admin |
| 12:24 PM (3 hours ago) | |||
| ||||
Hi Team,
I am encountering an issue when upgrading our PostgreSQL database from minor version 16.10 to 16.14.
During the upgrade, the database enters an indefinite recovery mode across the master, standby, and multiple cascading replicas. Due to legacy requirements, we are utilizing streaming replication for this setup. Interestingly, when performing the same migration to version 16.13, the process completes successfully without any issues.
Could you please confirm if there are any known bugs regarding this behavior in version 16.14?
For reference, please find the details of our test steps and logs below.
1. Upgrade Attempt to Version 16.14 (Database Stays in Recovery Mode)
Version Check:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# /usr/local/pgsql/bin/psql --version
psql (PostgreSQL) 16.14
Process Status:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ps -ef|grep rec
postgres 48 43 1 15:28 ? 00:00:01 postgres: startup recovering 0000000100002499000000E4
root 166 58 0 15:30 pts/0 00:00:00 grep --color=auto rec
WAL Directory Details:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -l 0000000100002499000000E4
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:25 0000000100002499000000E4
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -lt|more
total 55001312
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000059
drwxrws--- 2 postgres postgres 225280 Jul 2 14:54 archive_status
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000058
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000057
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000056
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000055
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000054
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000053
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000052
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000051
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000050
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004F
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004E
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004D
PostgreSQL Logs:
2026-07-02 15:28:38.709 UTC [43]: [3-1] user=,db= >LOG: starting PostgreSQL 16.14 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, 64-bit
2026-07-02 15:28:38.709 UTC [43]: [4-1] user=,db= >LOG: listening on IPv4 address "0.0.0.0", port 5432
2026-07-02 15:28:38.709 UTC [43]: [5-1] user=,db= >LOG: listening on IPv6 address "::", port 5432
2026-07-02 15:28:38.722 UTC [43]: [6-1] user=,db= >LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2026-07-02 15:28:38.753 UTC [48]: [1-1] user=,db= >LOG: database system was interrupted while in recovery at log time 2026-07-02 14:22:09 UTC
2026-07-02 15:28:38.753 UTC [48]: [2-1] user=,db= >HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
2026-07-02 15:28:38.979 UTC [48]: [3-1] user=,db= >LOG: entering standby mode
2026-07-02 15:28:38.993 UTC [48]: [4-1] user=,db= >LOG: redo starts at 2499/CFF32450
2026-07-02 15:28:47.282 UTC [49]: [1-1] user=127.0.0.1,db=postgres >FATAL: the database system is not yet accepting connections
2026-07-02 15:28:47.282 UTC [49]: [2-1] user=127.0.0.1,db=postgres >DETAIL: Consistent recovery state has not been yet reached.
2026-07-02 15:28:57.291 UTC [57]: [1-1] user=127.0.0.1,db=postgres >FATAL: the database system is not yet accepting connections
2026-07-02 15:28:57.291 UTC [57]: [2-1] user=127.0.0.1,db=postgres >DETAIL: Consistent recovery state has not been yet reached.
2. Successful Migration to Version 16.13 (Streaming Replication Working)
Version Check:
root@pgreplica-master-0:/# psql --version
psql (PostgreSQL) 16.13 (Debian 16.13-1.pgdg12+1)
Process Status (Walreceiver Streaming Active):
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ps -ef|grep rec
postgres 48 43 32 15:40 ? 00:00:20 postgres: startup recovering 000000010000249B0000002E
postgres 50 43 7 15:40 ? 00:00:03 postgres: walreceiver streaming 249B/2E660000
root 105 80 0 15:41 pts/0 00:00:00 grep rec
WAL File Check:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -lth 000000010000249B0000002E
-rw------- 1 postgres postgres 16M Jul 2 15:41 000000010000249B0000002E
Database Connection Verification:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# psql -U postgres -c " select version () "
version
------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 16.13 (Debian 16.13-1.pgdg12+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14+deb12u1) 12.2.0, 64-bit
(1 row)
Please let me know if any further details or logs are required to investigate this issue.
Best regards,
Manoj K P
I am encountering an issue when upgrading our PostgreSQL database from minor version 16.10 to 16.14.
During the upgrade, the database enters an indefinite recovery mode across the master, standby, and multiple cascading replicas. Due to legacy requirements, we are utilizing streaming replication for this setup. Interestingly, when performing the same migration to version 16.13, the process completes successfully without any issues.
Could you please confirm if there are any known bugs regarding this behavior in version 16.14?
For reference, please find the details of our test steps and logs below.
1. Upgrade Attempt to Version 16.14 (Database Stays in Recovery Mode)
Version Check:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# /usr/local/pgsql/bin/psql --version
psql (PostgreSQL) 16.14
Process Status:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ps -ef|grep rec
postgres 48 43 1 15:28 ? 00:00:01 postgres: startup recovering 0000000100002499000000E4
root 166 58 0 15:30 pts/0 00:00:00 grep --color=auto rec
WAL Directory Details:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -l 0000000100002499000000E4
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:25 0000000100002499000000E4
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -lt|more
total 55001312
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000059
drwxrws--- 2 postgres postgres 225280 Jul 2 14:54 archive_status
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000058
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000057
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000056
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:54 000000010000249A00000055
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000054
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000053
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000052
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000051
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A00000050
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004F
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004E
-rw-rw---- 1 postgres postgres 16777216 Jul 2 14:53 000000010000249A0000004D
PostgreSQL Logs:
2026-07-02 15:28:38.709 UTC [43]: [3-1] user=,db= >LOG: starting PostgreSQL 16.14 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, 64-bit
2026-07-02 15:28:38.709 UTC [43]: [4-1] user=,db= >LOG: listening on IPv4 address "0.0.0.0", port 5432
2026-07-02 15:28:38.709 UTC [43]: [5-1] user=,db= >LOG: listening on IPv6 address "::", port 5432
2026-07-02 15:28:38.722 UTC [43]: [6-1] user=,db= >LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2026-07-02 15:28:38.753 UTC [48]: [1-1] user=,db= >LOG: database system was interrupted while in recovery at log time 2026-07-02 14:22:09 UTC
2026-07-02 15:28:38.753 UTC [48]: [2-1] user=,db= >HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
2026-07-02 15:28:38.979 UTC [48]: [3-1] user=,db= >LOG: entering standby mode
2026-07-02 15:28:38.993 UTC [48]: [4-1] user=,db= >LOG: redo starts at 2499/CFF32450
2026-07-02 15:28:47.282 UTC [49]: [1-1] user=127.0.0.1,db=postgres >FATAL: the database system is not yet accepting connections
2026-07-02 15:28:47.282 UTC [49]: [2-1] user=127.0.0.1,db=postgres >DETAIL: Consistent recovery state has not been yet reached.
2026-07-02 15:28:57.291 UTC [57]: [1-1] user=127.0.0.1,db=postgres >FATAL: the database system is not yet accepting connections
2026-07-02 15:28:57.291 UTC [57]: [2-1] user=127.0.0.1,db=postgres >DETAIL: Consistent recovery state has not been yet reached.
2. Successful Migration to Version 16.13 (Streaming Replication Working)
Version Check:
root@pgreplica-master-0:/# psql --version
psql (PostgreSQL) 16.13 (Debian 16.13-1.pgdg12+1)
Process Status (Walreceiver Streaming Active):
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ps -ef|grep rec
postgres 48 43 32 15:40 ? 00:00:20 postgres: startup recovering 000000010000249B0000002E
postgres 50 43 7 15:40 ? 00:00:03 postgres: walreceiver streaming 249B/2E660000
root 105 80 0 15:41 pts/0 00:00:00 grep rec
WAL File Check:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# ls -lth 000000010000249B0000002E
-rw------- 1 postgres postgres 16M Jul 2 15:41 000000010000249B0000002E
Database Connection Verification:
root@pgreplica-master-0:/var/lib/postgresql/data/pg_wal# psql -U postgres -c " select version () "
version
------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 16.13 (Debian 16.13-1.pgdg12+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14+deb12u1) 12.2.0, 64-bit
(1 row)
Please let me know if any further details or logs are required to investigate this issue.
Best regards,
Manoj K P
pgsql-admin by date:
