Thread: [HACKERS] Reading backup label file for checkpoint and redo location duringcrash recovery

Greetings Satya,

* Satyanarayana Narlapuram (Satyanarayana.Narlapuram@microsoft.com) wrote:
> During crash recovery, last checkpoint record information is obtained from the backup label if present, instead of
gettingit from the control file. This behavior is causing PostgreSQL database cluster not to come up until the backup
labelfile is deleted (as the error message says). 
>
> if (checkPoint.redo < checkPointLoc)
>                       {
>                              if (!ReadRecord(xlogreader, checkPoint.redo, LOG, false))
>                                     ereport(FATAL,
>                                                   (errmsg("could not find redo location referenced by checkpoint
record"),
>                                                   errhint("If you are not restoring from a backup, try removing the
file\"%s/backup_label\".", DataDir))); 
>                       }
>
> If we are recovering from a dump file, reading from the backup label files makes sense as the control file could be
archivedafter a few checkpoints. But this is not the case for crash recovery, and is always safe to read the checkpoint
recordinformation from the control file. 
> Is this behavior kept this way as there is no clear way to distinguish between the recovery from the dump and the
regularcrash recovery? 

This is why the exclusive backup method has been deprecated in PG10 in
favor of the non-exclusive backup method, which avoids this by not
creating a backup label file (it's up to the backup software to store
the necessary information and create the file for use during recovery).

Please see:

https://www.postgresql.org/docs/10/static/continuous-archiving.html

In particular, section 25.3.3.

Thanks!

Stephen

On 2017-09-25 13:43:32 -0400, Stephen Frost wrote:
> Greetings Satya,
> 
> * Satyanarayana Narlapuram (Satyanarayana.Narlapuram@microsoft.com) wrote:
> > During crash recovery, last checkpoint record information is obtained from the backup label if present, instead of
gettingit from the control file. This behavior is causing PostgreSQL database cluster not to come up until the backup
labelfile is deleted (as the error message says).
 
> > 
> > if (checkPoint.redo < checkPointLoc)
> >                       {
> >                              if (!ReadRecord(xlogreader, checkPoint.redo, LOG, false))
> >                                     ereport(FATAL,
> >                                                   (errmsg("could not find redo location referenced by checkpoint
record"),
> >                                                   errhint("If you are not restoring from a backup, try removing the
file\"%s/backup_label\".", DataDir)));
 
> >                       }
> > 
> > If we are recovering from a dump file, reading from the backup label files makes sense as the control file could be
archivedafter a few checkpoints. But this is not the case for crash recovery, and is always safe to read the checkpoint
recordinformation from the control file.
 
> > Is this behavior kept this way as there is no clear way to distinguish between the recovery from the dump and the
regularcrash recovery?
 
> 
> This is why the exclusive backup method has been deprecated in PG10 in
> favor of the non-exclusive backup method, which avoids this by not
> creating a backup label file (it's up to the backup software to store
> the necessary information and create the file for use during recovery).
> 
> Please see:
> 
> https://www.postgresql.org/docs/10/static/continuous-archiving.html
> 
> In particular, section 25.3.3.

Might not be obvious for the more casual contributor:

And even before then, using pg_basebackup / the streaming replication
protocol version of creating a base backup, it was avoided.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Magnus Hagander (magnus@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > * Satyanarayana Narlapuram (Satyanarayana.Narlapuram@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is obtained
> > from the backup label if present, instead of getting it from the control
> > file. This behavior is causing PostgreSQL database cluster not to come up
> > until the backup label file is deleted (as the error message says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >                       {
> > >                              if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > >                                     ereport(FATAL,
> > >                                                   (errmsg("could not
> > find redo location referenced by checkpoint record"),
> > >                                                   errhint("If you are
> > not restoring from a backup, try removing the file \"%s/backup_label\".",
> > DataDir)));
> > >                       }
> > >
> > > If we are recovering from a dump file, reading from the backup label
> > files makes sense as the control file could be archived after a few
> > checkpoints. But this is not the case for crash recovery, and is always
> > safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10 in
> > favor of the non-exclusive backup method, which avoids this by not
> > creating a backup label file (it's up to the backup software to store
> > the necessary information and create the file for use during recovery).
>
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen

Thank you! Got it.

-----Original Message-----
From: Stephen Frost [mailto:sfrost@snowman.net]
Sent: Monday, September 25, 2017 10:57 AM
To: Magnus Hagander <magnus@hagander.net>
Cc: Satyanarayana Narlapuram <Satyanarayana.Narlapuram@microsoft.com>; PostgreSQL-development
<pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

* Magnus Hagander (magnus@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > * Satyanarayana Narlapuram (Satyanarayana.Narlapuram@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is
> > > obtained
> > from the backup label if present, instead of getting it from the
> > control file. This behavior is causing PostgreSQL database cluster
> > not to come up until the backup label file is deleted (as the error message says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >                       {
> > >                              if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > >                                     ereport(FATAL,
> > >                                                   (errmsg("could
> > > not
> > find redo location referenced by checkpoint record"),
> > >                                                   errhint("If you
> > > are
> > not restoring from a backup, try removing the file
> > \"%s/backup_label\".", DataDir)));
> > >                       }
> > >
> > > If we are recovering from a dump file, reading from the backup
> > > label
> > files makes sense as the control file could be archived after a few
> > checkpoints. But this is not the case for crash recovery, and is
> > always safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to
> > > distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10
> > in favor of the non-exclusive backup method, which avoids this by
> > not creating a backup label file (it's up to the backup software to
> > store the necessary information and create the file for use during recovery).
>
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers