Thread: Re: [HACKERS] Online backup vs Continuous backup

Re: [HACKERS] Online backup vs Continuous backup

From
Bruce Momjian
Date:
I used your suggestion and renamed "online backup" to "incremental
backup", and added a mention that many database vendors call it "online
backup".

Patch attached.

---------------------------------------------------------------------------

Rick Gigger wrote:
> How about:
>
> use "Online backup" or "Hot backup" to refer to either method of back
> since they are both done while the system is online or hot.
>
> If you want to get specific refer to doing a "sql dump" etc for using
> pg_dump
> Then use "Incremental backup" to refer to  the whole process of the
> WAL archival etc
> Refer to the actual log files themselves as transaction logs.
>
> That all seems to be pretty intuitive and non-ambiguous non-confusing
> to me.
>
> On Dec 26, 2005, at 11:44 AM, Tom Lane wrote:
>
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> I suggest the following patch to rename our capability "Continuous
> >> Backup".
> >
> > This doesn't seem like an improvement.  "Online backup" is the
> > standard
> > terminology AFAIK.
> >
> >             regards, tom lane
> >
> > ---------------------------(end of
> > broadcast)---------------------------
> > TIP 4: Have you searched our list archives?
> >
> >                http://archives.postgresql.org
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
Index: doc/src/sgml/backup.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/backup.sgml,v
retrieving revision 2.76
diff -c -c -r2.76 backup.sgml
*** doc/src/sgml/backup.sgml    7 Nov 2005 17:36:44 -0000    2.76
--- doc/src/sgml/backup.sgml    14 Feb 2006 04:00:50 -0000
***************
*** 19,25 ****
    <itemizedlist>
     <listitem><para><acronym>SQL</> dump</para></listitem>
     <listitem><para>File system level backup</para></listitem>
!    <listitem><para>On-line backup</para></listitem>
    </itemizedlist>
    Each has its own strengths and weaknesses.
   </para>
--- 19,25 ----
    <itemizedlist>
     <listitem><para><acronym>SQL</> dump</para></listitem>
     <listitem><para>File system level backup</para></listitem>
!    <listitem><para>Incremental backup</para></listitem>
    </itemizedlist>
    Each has its own strengths and weaknesses.
   </para>
***************
*** 372,382 ****
    </para>
   </sect1>

!  <sect1 id="backup-online">
!   <title>On-line backup and point-in-time recovery (PITR)</title>

    <indexterm zone="backup">
!    <primary>on-line backup</primary>
    </indexterm>

    <indexterm zone="backup">
--- 372,382 ----
    </para>
   </sect1>

!  <sect1 id="backup-incremental">
!   <title>Incremental backup and point-in-time recovery (PITR)</title>

    <indexterm zone="backup">
!    <primary>incremental backup</primary>
    </indexterm>

    <indexterm zone="backup">
***************
*** 452,458 ****
    </para>

    <para>
!    To recover successfully using an on-line backup, you need a continuous
     sequence of archived WAL files that extends back at least as far as the
     start time of your backup.  So to get started, you should set up and test
     your procedure for archiving WAL files <emphasis>before</> you take your
--- 452,459 ----
    </para>

    <para>
!    To recover successfully using an incremental backup (also called "online
!    backup" by many database vendors), you need a continuous
     sequence of archived WAL files that extends back at least as far as the
     start time of your backup.  So to get started, you should set up and test
     your procedure for archiving WAL files <emphasis>before</> you take your
***************
*** 782,793 ****
      <function>pg_start_backup</> or <function>pg_stop_backup</>, and
      you will therefore be left to your own devices to keep track of which
      backup dump is which and how far back the associated WAL files go.
!     It is generally better to follow the on-line backup procedure above.
     </para>
    </sect2>

    <sect2 id="backup-pitr-recovery">
!    <title>Recovering with an On-line Backup</title>

     <para>
      Okay, the worst has happened and you need to recover from your backup.
--- 783,794 ----
      <function>pg_start_backup</> or <function>pg_stop_backup</>, and
      you will therefore be left to your own devices to keep track of which
      backup dump is which and how far back the associated WAL files go.
!     It is generally better to follow the incremental backup procedure above.
     </para>
    </sect2>

    <sect2 id="backup-pitr-recovery">
!    <title>Recovering with an Incremental Backup</title>

     <para>
      Okay, the worst has happened and you need to recover from your backup.
***************
*** 1119,1129 ****
     </para>
    </sect2>

!   <sect2 id="backup-online-caveats">
     <title>Caveats</title>

     <para>
!     At this writing, there are several limitations of the on-line backup
      technique.  These will probably be fixed in future releases:

    <itemizedlist>
--- 1120,1130 ----
     </para>
    </sect2>

!   <sect2 id="backup-incremental-caveats">
     <title>Caveats</title>

     <para>
!     At this writing, there are several limitations of the incremental backup
      technique.  These will probably be fixed in future releases:

    <itemizedlist>
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.47
diff -c -c -r1.47 config.sgml
*** doc/src/sgml/config.sgml    5 Feb 2006 18:19:14 -0000    1.47
--- doc/src/sgml/config.sgml    14 Feb 2006 04:00:53 -0000
***************
*** 1387,1393 ****
         <para>
          Turning off this parameter does not affect use of
          WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="backup-online">).
         </para>

         <para>
--- 1387,1393 ----
         <para>
          Turning off this parameter does not affect use of
          WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="backup-incremental">).
         </para>

         <para>
Index: doc/src/sgml/func.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/func.sgml,v
retrieving revision 1.306
diff -c -c -r1.306 func.sgml
*** doc/src/sgml/func.sgml    12 Feb 2006 04:44:15 -0000    1.306
--- doc/src/sgml/func.sgml    14 Feb 2006 04:00:59 -0000
***************
*** 9784,9790 ****

     <para>
      For details about proper usage of these functions, see
!     <xref linkend="backup-online">.
     </para>

     <para>
--- 9784,9790 ----

     <para>
      For details about proper usage of these functions, see
!     <xref linkend="backup-incremental">.
     </para>

     <para>
Index: doc/src/sgml/wal.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v
retrieving revision 1.38
diff -c -c -r1.38 wal.sgml
*** doc/src/sgml/wal.sgml    4 Nov 2005 23:14:02 -0000    1.38
--- doc/src/sgml/wal.sgml    14 Feb 2006 04:01:00 -0000
***************
*** 136,142 ****
     <para>
      <acronym>WAL</acronym> also makes it possible to support on-line
      backup and point-in-time recovery, as described in <xref
!     linkend="backup-online">.  By archiving the WAL data we can support
      reverting to any time instant covered by the available WAL data:
      we simply install a prior physical backup of the database, and
      replay the WAL log just as far as the desired time.  What's more,
--- 136,142 ----
     <para>
      <acronym>WAL</acronym> also makes it possible to support on-line
      backup and point-in-time recovery, as described in <xref
!     linkend="backup-incremental">.  By archiving the WAL data we can support
      reverting to any time instant covered by the available WAL data:
      we simply install a prior physical backup of the database, and
      replay the WAL log just as far as the desired time.  What's more,

Re: [HACKERS] Online backup vs Continuous backup

From
Peter Eisentraut
Date:
Bruce Momjian wrote:
> I used your suggestion and renamed "online backup" to "incremental
> backup", and added a mention that many database vendors call it
> "online backup".

Consistency would then demand that the other two be renamed to "full
backup".  I think we had better suggestions earlier.


--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Re: [HACKERS] Online backup vs Continuous backup

From
Bruce Momjian
Date:
Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I used your suggestion and renamed "online backup" to "incremental
> > backup", and added a mention that many database vendors call it
> > "online backup".
>
> Consistency would then demand that the other two be renamed to "full
> backup".  I think we had better suggestions earlier.

I am also now thinking the word "incremental" is wrong because it
implies something that happens in parts, like "backup everything that
changed from last night until now" which not how this feature works.

I am thinking "Continuous Archiving" is the proper wording, and it
avoids the "backup" word.

Updated patch attached.

--
  Bruce Momjian   http://candle.pha.pa.us
  SRA OSS, Inc.   http://www.sraoss.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/backup.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/backup.sgml,v
retrieving revision 2.76
diff -c -c -r2.76 backup.sgml
*** doc/src/sgml/backup.sgml    7 Nov 2005 17:36:44 -0000    2.76
--- doc/src/sgml/backup.sgml    14 Feb 2006 04:00:50 -0000
***************
*** 19,25 ****
    <itemizedlist>
     <listitem><para><acronym>SQL</> dump</para></listitem>
     <listitem><para>File system level backup</para></listitem>
!    <listitem><para>On-line backup</para></listitem>
    </itemizedlist>
    Each has its own strengths and weaknesses.
   </para>
--- 19,25 ----
    <itemizedlist>
     <listitem><para><acronym>SQL</> dump</para></listitem>
     <listitem><para>File system level backup</para></listitem>
!    <listitem><para>Continuous Archiving</para></listitem>
    </itemizedlist>
    Each has its own strengths and weaknesses.
   </para>
***************
*** 372,382 ****
    </para>
   </sect1>

!  <sect1 id="backup-online">
!   <title>On-line backup and point-in-time recovery (PITR)</title>

    <indexterm zone="backup">
!    <primary>on-line backup</primary>
    </indexterm>

    <indexterm zone="backup">
--- 372,382 ----
    </para>
   </sect1>

!  <sect1 id="continuous-archiving">
!   <title>Continuous Archiving and Point-In-Time Recovery (PITR)</title>

    <indexterm zone="backup">
!    <primary>continuous archiving</primary>
    </indexterm>

    <indexterm zone="backup">
***************
*** 452,458 ****
    </para>

    <para>
!    To recover successfully using an on-line backup, you need a continuous
     sequence of archived WAL files that extends back at least as far as the
     start time of your backup.  So to get started, you should set up and test
     your procedure for archiving WAL files <emphasis>before</> you take your
--- 452,459 ----
    </para>

    <para>
!    To recover successfully using continuous archiving (also called "online
!    backup" by many database vendors), you need a continuous
     sequence of archived WAL files that extends back at least as far as the
     start time of your backup.  So to get started, you should set up and test
     your procedure for archiving WAL files <emphasis>before</> you take your
***************
*** 782,793 ****
      <function>pg_start_backup</> or <function>pg_stop_backup</>, and
      you will therefore be left to your own devices to keep track of which
      backup dump is which and how far back the associated WAL files go.
!     It is generally better to follow the on-line backup procedure above.
     </para>
    </sect2>

    <sect2 id="backup-pitr-recovery">
!    <title>Recovering with an On-line Backup</title>

     <para>
      Okay, the worst has happened and you need to recover from your backup.
--- 783,794 ----
      <function>pg_start_backup</> or <function>pg_stop_backup</>, and
      you will therefore be left to your own devices to keep track of which
      backup dump is which and how far back the associated WAL files go.
!     It is generally better to follow the continuous archiving procedure above.
     </para>
    </sect2>

    <sect2 id="backup-pitr-recovery">
!    <title>Recovering using a Continuous Archive Backup</title>

     <para>
      Okay, the worst has happened and you need to recover from your backup.
***************
*** 1119,1129 ****
     </para>
    </sect2>

!   <sect2 id="backup-online-caveats">
     <title>Caveats</title>

     <para>
!     At this writing, there are several limitations of the on-line backup
      technique.  These will probably be fixed in future releases:

    <itemizedlist>
--- 1120,1130 ----
     </para>
    </sect2>

!   <sect2 id="continuous-archiving-caveats">
     <title>Caveats</title>

     <para>
!     At this writing, there are several limitations of the continuous archiving
      technique.  These will probably be fixed in future releases:

    <itemizedlist>
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.47
diff -c -c -r1.47 config.sgml
*** doc/src/sgml/config.sgml    5 Feb 2006 18:19:14 -0000    1.47
--- doc/src/sgml/config.sgml    14 Feb 2006 04:00:53 -0000
***************
*** 1387,1393 ****
         <para>
          Turning off this parameter does not affect use of
          WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="backup-online">).
         </para>

         <para>
--- 1387,1393 ----
         <para>
          Turning off this parameter does not affect use of
          WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="continuous-archiving">).
         </para>

         <para>
Index: doc/src/sgml/func.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/func.sgml,v
retrieving revision 1.306
diff -c -c -r1.306 func.sgml
*** doc/src/sgml/func.sgml    12 Feb 2006 04:44:15 -0000    1.306
--- doc/src/sgml/func.sgml    14 Feb 2006 04:00:59 -0000
***************
*** 9784,9790 ****

     <para>
      For details about proper usage of these functions, see
!     <xref linkend="backup-online">.
     </para>

     <para>
--- 9784,9790 ----

     <para>
      For details about proper usage of these functions, see
!     <xref linkend="continuous-archiving">.
     </para>

     <para>
Index: doc/src/sgml/wal.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v
retrieving revision 1.38
diff -c -c -r1.38 wal.sgml
*** doc/src/sgml/wal.sgml    4 Nov 2005 23:14:02 -0000    1.38
--- doc/src/sgml/wal.sgml    14 Feb 2006 04:01:00 -0000
***************
*** 136,142 ****
     <para>
      <acronym>WAL</acronym> also makes it possible to support on-line
      backup and point-in-time recovery, as described in <xref
!     linkend="backup-online">.  By archiving the WAL data we can support
      reverting to any time instant covered by the available WAL data:
      we simply install a prior physical backup of the database, and
      replay the WAL log just as far as the desired time.  What's more,
--- 136,142 ----
     <para>
      <acronym>WAL</acronym> also makes it possible to support on-line
      backup and point-in-time recovery, as described in <xref
!     linkend="continuous-archiving">.  By archiving the WAL data we can support
      reverting to any time instant covered by the available WAL data:
      we simply install a prior physical backup of the database, and
      replay the WAL log just as far as the desired time.  What's more,

Re: [PATCHES] [HACKERS] Online backup vs Continuous backup

From
Bruce Momjian
Date:
Applied.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > I used your suggestion and renamed "online backup" to "incremental
> > > backup", and added a mention that many database vendors call it
> > > "online backup".
> >
> > Consistency would then demand that the other two be renamed to "full
> > backup".  I think we had better suggestions earlier.
>
> I am also now thinking the word "incremental" is wrong because it
> implies something that happens in parts, like "backup everything that
> changed from last night until now" which not how this feature works.
>
> I am thinking "Continuous Archiving" is the proper wording, and it
> avoids the "backup" word.
>
> Updated patch attached.
>
> --
>   Bruce Momjian   http://candle.pha.pa.us
>   SRA OSS, Inc.   http://www.sraoss.com
>
>   + If your life is a hard drive, Christ can be your backup. +

> Index: doc/src/sgml/backup.sgml
> ===================================================================
> RCS file: /cvsroot/pgsql/doc/src/sgml/backup.sgml,v
> retrieving revision 2.76
> diff -c -c -r2.76 backup.sgml
> *** doc/src/sgml/backup.sgml    7 Nov 2005 17:36:44 -0000    2.76
> --- doc/src/sgml/backup.sgml    14 Feb 2006 04:00:50 -0000
> ***************
> *** 19,25 ****
>     <itemizedlist>
>      <listitem><para><acronym>SQL</> dump</para></listitem>
>      <listitem><para>File system level backup</para></listitem>
> !    <listitem><para>On-line backup</para></listitem>
>     </itemizedlist>
>     Each has its own strengths and weaknesses.
>    </para>
> --- 19,25 ----
>     <itemizedlist>
>      <listitem><para><acronym>SQL</> dump</para></listitem>
>      <listitem><para>File system level backup</para></listitem>
> !    <listitem><para>Continuous Archiving</para></listitem>
>     </itemizedlist>
>     Each has its own strengths and weaknesses.
>    </para>
> ***************
> *** 372,382 ****
>     </para>
>    </sect1>
>
> !  <sect1 id="backup-online">
> !   <title>On-line backup and point-in-time recovery (PITR)</title>
>
>     <indexterm zone="backup">
> !    <primary>on-line backup</primary>
>     </indexterm>
>
>     <indexterm zone="backup">
> --- 372,382 ----
>     </para>
>    </sect1>
>
> !  <sect1 id="continuous-archiving">
> !   <title>Continuous Archiving and Point-In-Time Recovery (PITR)</title>
>
>     <indexterm zone="backup">
> !    <primary>continuous archiving</primary>
>     </indexterm>
>
>     <indexterm zone="backup">
> ***************
> *** 452,458 ****
>     </para>
>
>     <para>
> !    To recover successfully using an on-line backup, you need a continuous
>      sequence of archived WAL files that extends back at least as far as the
>      start time of your backup.  So to get started, you should set up and test
>      your procedure for archiving WAL files <emphasis>before</> you take your
> --- 452,459 ----
>     </para>
>
>     <para>
> !    To recover successfully using continuous archiving (also called "online
> !    backup" by many database vendors), you need a continuous
>      sequence of archived WAL files that extends back at least as far as the
>      start time of your backup.  So to get started, you should set up and test
>      your procedure for archiving WAL files <emphasis>before</> you take your
> ***************
> *** 782,793 ****
>       <function>pg_start_backup</> or <function>pg_stop_backup</>, and
>       you will therefore be left to your own devices to keep track of which
>       backup dump is which and how far back the associated WAL files go.
> !     It is generally better to follow the on-line backup procedure above.
>      </para>
>     </sect2>
>
>     <sect2 id="backup-pitr-recovery">
> !    <title>Recovering with an On-line Backup</title>
>
>      <para>
>       Okay, the worst has happened and you need to recover from your backup.
> --- 783,794 ----
>       <function>pg_start_backup</> or <function>pg_stop_backup</>, and
>       you will therefore be left to your own devices to keep track of which
>       backup dump is which and how far back the associated WAL files go.
> !     It is generally better to follow the continuous archiving procedure above.
>      </para>
>     </sect2>
>
>     <sect2 id="backup-pitr-recovery">
> !    <title>Recovering using a Continuous Archive Backup</title>
>
>      <para>
>       Okay, the worst has happened and you need to recover from your backup.
> ***************
> *** 1119,1129 ****
>      </para>
>     </sect2>
>
> !   <sect2 id="backup-online-caveats">
>      <title>Caveats</title>
>
>      <para>
> !     At this writing, there are several limitations of the on-line backup
>       technique.  These will probably be fixed in future releases:
>
>     <itemizedlist>
> --- 1120,1130 ----
>      </para>
>     </sect2>
>
> !   <sect2 id="continuous-archiving-caveats">
>      <title>Caveats</title>
>
>      <para>
> !     At this writing, there are several limitations of the continuous archiving
>       technique.  These will probably be fixed in future releases:
>
>     <itemizedlist>
> Index: doc/src/sgml/config.sgml
> ===================================================================
> RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
> retrieving revision 1.47
> diff -c -c -r1.47 config.sgml
> *** doc/src/sgml/config.sgml    5 Feb 2006 18:19:14 -0000    1.47
> --- doc/src/sgml/config.sgml    14 Feb 2006 04:00:53 -0000
> ***************
> *** 1387,1393 ****
>          <para>
>           Turning off this parameter does not affect use of
>           WAL archiving for point-in-time recovery (PITR)
> !         (see <xref linkend="backup-online">).
>          </para>
>
>          <para>
> --- 1387,1393 ----
>          <para>
>           Turning off this parameter does not affect use of
>           WAL archiving for point-in-time recovery (PITR)
> !         (see <xref linkend="continuous-archiving">).
>          </para>
>
>          <para>
> Index: doc/src/sgml/func.sgml
> ===================================================================
> RCS file: /cvsroot/pgsql/doc/src/sgml/func.sgml,v
> retrieving revision 1.306
> diff -c -c -r1.306 func.sgml
> *** doc/src/sgml/func.sgml    12 Feb 2006 04:44:15 -0000    1.306
> --- doc/src/sgml/func.sgml    14 Feb 2006 04:00:59 -0000
> ***************
> *** 9784,9790 ****
>
>      <para>
>       For details about proper usage of these functions, see
> !     <xref linkend="backup-online">.
>      </para>
>
>      <para>
> --- 9784,9790 ----
>
>      <para>
>       For details about proper usage of these functions, see
> !     <xref linkend="continuous-archiving">.
>      </para>
>
>      <para>
> Index: doc/src/sgml/wal.sgml
> ===================================================================
> RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v
> retrieving revision 1.38
> diff -c -c -r1.38 wal.sgml
> *** doc/src/sgml/wal.sgml    4 Nov 2005 23:14:02 -0000    1.38
> --- doc/src/sgml/wal.sgml    14 Feb 2006 04:01:00 -0000
> ***************
> *** 136,142 ****
>      <para>
>       <acronym>WAL</acronym> also makes it possible to support on-line
>       backup and point-in-time recovery, as described in <xref
> !     linkend="backup-online">.  By archiving the WAL data we can support
>       reverting to any time instant covered by the available WAL data:
>       we simply install a prior physical backup of the database, and
>       replay the WAL log just as far as the desired time.  What's more,
> --- 136,142 ----
>      <para>
>       <acronym>WAL</acronym> also makes it possible to support on-line
>       backup and point-in-time recovery, as described in <xref
> !     linkend="continuous-archiving">.  By archiving the WAL data we can support
>       reverting to any time instant covered by the available WAL data:
>       we simply install a prior physical backup of the database, and
>       replay the WAL log just as far as the desired time.  What's more,

>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq

--
  Bruce Momjian   http://candle.pha.pa.us
  SRA OSS, Inc.   http://www.sraoss.com

  + If your life is a hard drive, Christ can be your backup. +