Thread: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

[PERFORM] How Can I check PostgreSQL backup is successfully or not ?

From
Dinesh Chandra 12108
Date:

Hi,

 

We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.

 

How can I check my yesterday backup is successfully or not?

Is there any query or view by which I can check it?

 

Regards,

Dinesh Chandra

|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.

------------------------------------------------------------------

Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com

Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.

 




DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Rick Otten
Date:
Although it doesn't really tell if the pg_dump was successful (you'll need to do a full restore to be sure), I generate an archive list.  If that fails, the backup clearly wasn't successful, and if it succeeds, odds are pretty good that it worked:

-- bash code snippet --
archiveList=`pg_restore -l ${backupFolder}`
if [[ ! ${archiveList} =~ "Archive created at" ]]
then
    echo "PostgreSQL backup - Archive List Test Failed for ${hostName}:${dbName}"
    echo "Archive listing:"
    echo ${archiveList}
    exit 1
fi
-----------------------



On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:

Hi,

 

We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.

 

How can I check my yesterday backup is successfully or not?

Is there any query or view by which I can check it?

 

Regards,

Dinesh Chandra

|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.

------------------------------------------------------------------

Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com

Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.

 




DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
John Gorman
Date:

Even though it's not listed in any of the documentation or “pg_dump --help” you can check the return code of the process. A return code greater than 0 (zero) usually indicates a failure

 

./bin >pg_dump -U dummy_user  dummy_database; echo $?

1

 

From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Rick Otten
Sent: Monday, February 27, 2017 3:36 AM
To: Dinesh Chandra 12108
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

 

Although it doesn't really tell if the pg_dump was successful (you'll need to do a full restore to be sure), I generate an archive list.  If that fails, the backup clearly wasn't successful, and if it succeeds, odds are pretty good that it worked:

 

-- bash code snippet --

archiveList=`pg_restore -l ${backupFolder}`

if [[ ! ${archiveList} =~ "Archive created at" ]]

then

    echo "PostgreSQL backup - Archive List Test Failed for ${hostName}:${dbName}"

    echo "Archive listing:"

    echo ${archiveList}

    exit 1

fi

-----------------------

 

 

 

On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:

Hi,

 

We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.

 

How can I check my yesterday backup is successfully or not?

Is there any query or view by which I can check it?

 

Regards,

Dinesh Chandra

|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.

------------------------------------------------------------------

Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com

Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.

 

 



DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

 

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a
perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad
backup?
And can #2 fail on a good database, and if so, can you detect that?




Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Dinesh Chandra 12108
Date:
Hi,

I run the below command
[postgres@CPPMoma-DB05 bin]$ pg_dump -U postgres  moma_ts_oce; echo $

Output was like this(Last few lines )
------------------------------
-- Name: public; Type: ACL; Schema: -; Owner: postgres
--

REVOKE ALL ON SCHEMA public FROM PUBLIC;
REVOKE ALL ON SCHEMA public FROM postgres;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO PUBLIC;

--
-- PostgreSQL database dump complete
--

$

Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.
------------------------------------------------------------------
Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com
Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>;
pgsql-performance@postgresql.org;pgsql-performance-owner@postgresql.org
 
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong
database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?



________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged
information.Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended
recipient,please contact the sender by reply email and destroy all copies of the original message. Check all
attachmentsfor viruses before opening them. All views or opinions presented in this e-mail are those of the author and
maynot reflect the opinion of Cyient or those of our affiliates.
 

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Dinesh Chandra 12108
Date:
Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>;
pgsql-performance@postgresql.org;pgsql-performance-owner@postgresql.org
 
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong
database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?



________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged
information.Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended
recipient,please contact the sender by reply email and destroy all copies of the original message. Check all
attachmentsfor viruses before opening them. All views or opinions presented in this e-mail are those of the author and
maynot reflect the opinion of Cyient or those of our affiliates.
 

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Rick Otten
Date:
This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk.  By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.

The best way to ensure good backups is to establish a regular practice of restoring a backup to another database.  The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis.  Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.

pg_dump uses the ordinary postgresql COPY command to extract data from the tables.  Beyond that, I'm not sure how it works.  Sorry I can't help you there.


On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?



________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Dinesh Chandra 12108
Date:

Dear Rick,

 

Thanks for your valuable reply.

 

But the daily restoration of backup to another database is really so time consuming because our databases size is greater than 2TB.

 

Regards,

Dinesh Chandra

|Database administrator (Oracle/PostgreSQL)|

 

From: Rick Otten [mailto:rottenwindfish@gmail.com]
Sent: 02 March, 2017 6:45 PM
To: Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

 

This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk.  By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.

 

The best way to ensure good backups is to establish a regular practice of restoring a backup to another database.  The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis.  Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.

 

pg_dump uses the ordinary postgresql COPY command to extract data from the tables.  Beyond that, I'm not sure how it works.  Sorry I can't help you there.

 

 

On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:

Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org

Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?


________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

 

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
"ldh@laurent-hasson.com"
Date:
It'd be so nice to have some checks to guarantee the backup is trustworthy. Restoring the db is imho not a very good option in general:
 - large databases are a problem. My db is about 3TB. Time plus disk space is a big blocker.
 - also, what if the backup is incomplete? Just restoring the db successfully is not enough right? You'd have to compare with the prod to make sure nothing was missed... in a fast moving outfit where the db today will have tons of new/changed deleted stuff from yesterday.. how to even do that?

I am in a warehouse environment, so I have given ‎up on guaranteeing backups and in a case of trouble, i'll spend 20h rebuilding my db. So I have a way out but i'd much prefer working with trustworthy backups.


Sent from my BlackBerry 10 smartphone.
From: Rick Otten
Sent: Thursday, March 2, 2017 08:19
To: Dinesh Chandra 12108
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk.  By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.

The best way to ensure good backups is to establish a regular practice of restoring a backup to another database.  The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis.  Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.

pg_dump uses the ordinary postgresql COPY command to extract data from the tables.  Beyond that, I'm not sure how it works.  Sorry I can't help you there.


On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?



________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Dinesh Chandra 12108
Date:

May you please share what types of check if there to guarantee the backup is trustworthy.  

 

Regards,

Dinesh Chandra

|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.

 

From: ldh@laurent-hasson.com [mailto:ldh@laurent-hasson.com]
Sent: 02 March, 2017 9:57 PM
To: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

 

It'd be so nice to have some checks to guarantee the backup is trustworthy. Restoring the db is imho not a very good option in general:

 - large databases are a problem. My db is about 3TB. Time plus disk space is a big blocker.

 - also, what if the backup is incomplete? Just restoring the db successfully is not enough right? You'd have to compare with the prod to make sure nothing was missed... in a fast moving outfit where the db today will have tons of new/changed deleted stuff from yesterday.. how to even do that?

 

I am in a warehouse environment, so I have given ‎up on guaranteeing backups and in a case of trouble, i'll spend 20h rebuilding my db. So I have a way out but i'd much prefer working with trustworthy backups.

 

 

Sent from my BlackBerry 10 smartphone.

From: Rick Otten

Sent: Thursday, March 2, 2017 08:19

To: Dinesh Chandra 12108

Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

 

This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk.  By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.

 

The best way to ensure good backups is to establish a regular practice of restoring a backup to another database.  The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis.  Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.

 

pg_dump uses the ordinary postgresql COPY command to extract data from the tables.  Beyond that, I'm not sure how it works.  Sorry I can't help you there.

 

 

On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:

Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org

Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?


________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

 

Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?

From
Gustavo Vargas
Date:
Hi Rick, 

You Always have a doubt, if backup is full and done.

In my case, I have one Client using a large database, and the base in mysql, an when we stop traffic about 8:00PM  start a lot of routines. I have about 2hours of Exclusive lock.

For solve one problem like these in this Client. We prepared a replication on other servers, and doing a backup in a replications servers.

In My Opinion Postgres Replication work better way than mysql.


Pros.:  You have a second and third server and (n).
Can compare data and have a control of replication log.
And dont need do a directly backup in production enviroment

Best Regards



Gustavo Neves Vargas

+55(41) 9157-7816
+55(41) 3058-4967




On 2 Mar 2017, at 13:26, ldh@laurent-hasson.com wrote:

It'd be so nice to have some checks to guarantee the backup is trustworthy. Restoring the db is imho not a very good option in general:
 - large databases are a problem. My db is about 3TB. Time plus disk space is a big blocker.
 - also, what if the backup is incomplete? Just restoring the db successfully is not enough right? You'd have to compare with the prod to make sure nothing was missed... in a fast moving outfit where the db today will have tons of new/changed deleted stuff from yesterday.. how to even do that?

I am in a warehouse environment, so I have given ‎up on guaranteeing backups and in a case of trouble, i'll spend 20h rebuilding my db. So I have a way out but i'd much prefer working with trustworthy backups.


Sent from my BlackBerry 10 smartphone.
From: Rick Otten
Sent: Thursday, March 2, 2017 08:19
To: Dinesh Chandra 12108
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk.  By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.

The best way to ensure good backups is to establish a regular practice of restoring a backup to another database.  The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis.  Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.

pg_dump uses the ordinary postgresql COPY command to extract data from the tables.  Beyond that, I'm not sure how it works.  Sorry I can't help you there.


On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,

When I issue the bleow command
  > ./bin >pg_dump -U dummy_user  dummy_database; echo $?

I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??

Regards,
Dinesh Chandra

-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?

On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user  dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list.  If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.


It's important to note the distinction between

"the backup process did not fail"

and

"we now have a trustworthy backup"

And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.

So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?

What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?



________________________________

DISCLAIMER:

This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance