I have a 1 primary, 2 standby streaming replication cluster. The primary node (`pgsql/16-b/data`) shut down to trigger a failover. One standby (`psql/16-a/data`) was promoted to primary, and write operations were performed on the new primary and replicated to the other standby. I attempted to use pg_rewind to sync the failed primary with the current primary, but it failed. The error shows pg_rewind is trying to read from an incorrect WAL file `000000010000000000000015` instead of `000000040000000000000015` as per the control data:
pg_rewind: servers diverged at WAL location 150000A0/0 on timeline 4 pg_rewind: error: could not open file "/var/lib/pgsql/16-b/data/pg_wal/000000010000000000000015": No such file or directory pg_rewind: error: could not read WAL record at 0/15000028
Failed Primary Control Data: pg_control version number: 1300 Catalog version number: 202307071 Database system identifier: 7389684959150618064 Database cluster state: shut down pg_control last modified: Sat 13 Jul 2024 08:50:32 AM +04 Latest checkpoint location: 0/15000028 Latest checkpoint's REDO location: 0/15000028 Latest checkpoint's REDO WAL file: 000000040000000000000015 Latest checkpoint's TimeLineID: 4 Latest checkpoint's PrevTimeLineID: 4
Current Primary Control Data: pg_control version number: 1300 Catalog version number: 202307071 Database system identifier: 7389684959150618064 Database cluster state: in production pg_control last modified: Sat 13 Jul 2024 12:09:43 PM +04 Latest checkpoint location: 0/15009590 Latest checkpoint's REDO location: 0/15009558 Latest checkpoint's REDO WAL file: 000000050000000000000015 Latest checkpoint's TimeLineID: 5 Latest checkpoint's PrevTimeLineID: 5
Could anyone provide guidance on resolving this issue? I cannot find the WAL file that pg_rewind is trying to open and read in the data/pg_wal directory or my WAL archive directory.
