Re: Checkpoint Err on Startup of Rsynced System - Mailing list pgsql-general

From Scott Mead
Subject Re: Checkpoint Err on Startup of Rsynced System
Date
Msg-id CAKq0gvL2SuNfXLCm=AhvqW0o4HvtR-iBOvJtH5oceP6vftCe6g@mail.gmail.com
Whole thread Raw
In response to Checkpoint Err on Startup of Rsynced System  (Jim Longwill <longwill@psmfc.org>)
Responses Re: Checkpoint Err on Startup of Rsynced System
List pgsql-general


On Tue, May 31, 2016 at 1:13 PM, Jim Longwill <longwill@psmfc.org> wrote:
I am trying to setup a 2nd, identical, db server (M2) for development and I've run into a problem with starting up the 2nd Postgres installation.

Here's what I've done:
  1) did a 'clone' of 1st (production) machine M1 (so both machines on Cent OS 7.2)
  2) setup an rsync operation, did a complete 'rsync' from M1 to M2
  3) did a final 'CHECKPOINT' command on M1 postgres
  4) shutdown postgres on M1 with 'pg_ctl stop'
  5) did final 'rsync' operation  (then restarted postgres on M1 with 'pg_ctl start')
  6) tried to startup postgres on M2

It won't start, & in the log file gives the error message:
...
< 2016-05-31 09:02:52.337 PDT >LOG:  invalid primary checkpoint record
< 2016-05-31 09:02:52.337 PDT >LOG:  invalid secondary checkpoint record
< 2016-05-31 09:02:52.337 PDT >PANIC:  could not locate a valid checkpoint record
< 2016-05-31 09:02:53.184 PDT >LOG:  startup process (PID 26680) was terminated by signal 6: Aborted
< 2016-05-31 09:02:53.184 PDT >LOG:  aborting startup due to startup process failure

I've tried several times to do this but always get this result.  So, do I need to do a new 'initdb..' operation on machine M2 + restore from M1 backups?  Or is there another way to fix this?

You should have stopped M1 prior to taking the backup.  If you can't do that, it can be done online via:
  1. Setup archiving
  2. select pg_start_backup('some label'); 
  3. <run rsync>
  4. select pg_stop_backup();
  Without archiving and the pg_[start|stop]_backup, you're not guaranteed anything.  You could use an atomic snapshot (LVM, storage, etc...), but it's got to be a true snapshot.  Without that, you need archiving + start / stop backup.


--Scott


--o--o--o--o--o--o--o--o--o--o--o--o--
Jim Longwill
PSMFC Regional Mark Processing Center
JLongwill@psmfc.org
--o--o--o--o--o--o--o--o--o--o--o--o--


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general



--
--
Scott Mead
Sr. Architect
OpenSCG

pgsql-general by date:

Previous
From: Jim Longwill
Date:
Subject: Checkpoint Err on Startup of Rsynced System
Next
From: Joseph Kregloh
Date:
Subject: Log Shipping