Thread: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
[PERFORM] How Can I check PostgreSQL backup is successfully or not ?
Hi,
We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.
How can I check my yesterday backup is successfully or not?
Is there any query or view by which I can check it?
Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.
------------------------------------------------------------------
Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com
Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
Hi,
We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.
How can I check my yesterday backup is successfully or not?
Is there any query or view by which I can check it?
Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.
------------------------------
------------------------------ ------ Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com
Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
Even though it's not listed in any of the documentation or “pg_dump --help” you can check the return code of the process. A return code greater than 0 (zero) usually indicates a failure
./bin >pg_dump -U dummy_user dummy_database; echo $?
1
From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Rick Otten
Sent: Monday, February 27, 2017 3:36 AM
To: Dinesh Chandra 12108
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
Although it doesn't really tell if the pg_dump was successful (you'll need to do a full restore to be sure), I generate an archive list. If that fails, the backup clearly wasn't successful, and if it succeeds, odds are pretty good that it worked:
-- bash code snippet --
archiveList=`pg_restore -l ${backupFolder}`
if [[ ! ${archiveList} =~ "Archive created at" ]]
then
echo "PostgreSQL backup - Archive List Test Failed for ${hostName}:${dbName}"
echo "Archive listing:"
echo ${archiveList}
exit 1
fi
-----------------------
On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,
We are taking daily full backup of PostgreSQL database using PG_DUMP which is automatic scheduled through Cronjobs.
How can I check my yesterday backup is successfully or not?
Is there any query or view by which I can check it?
Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.
------------------------------------------------------------------
Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com
Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India.
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
On 2017-02-27 14:29, John Gorman wrote: > Even though it's not listed in any of the documentation or “pg_dump > --help” you can check the return code of the process. A return code > greater than 0 (zero) usually indicates a failure > > ./bin >pg_dump -U dummy_user dummy_database; echo $? > > 1 > > FROM: pgsql-performance-owner@postgresql.org > [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick > Otten > SENT: Monday, February 27, 2017 3:36 AM > TO: Dinesh Chandra 12108 > CC: pgsql-performance@postgresql.org > SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is > successfully or not ? > > Although it doesn't really tell if the pg_dump was successful (you'll > need to do a full restore to be sure), I generate an archive list. If > that fails, the backup clearly wasn't successful, and if it succeeds, > odds are pretty good that it worked: > > On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 > <Dinesh.Chandra@cyient.com> wrote: > > Hi, > > We are taking daily full backup of PostgreSQL database using PG_DUMP > which is automatic scheduled through Cronjobs. > > How can I check my yesterday backup is successfully or not? > > Is there any query or view by which I can check it? > > REGARDS, > > DINESH CHANDRA > > |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA. It's important to note the distinction between "the backup process did not fail" and "we now have a trustworthy backup" And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database. So what is it that you want to make sure of: 1. Did the process give an error? 2. Did the process create a usable backup? What are the chances of #1 reporting success but still producing a bad backup? And can #2 fail on a good database, and if so, can you detect that?
Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?
Hi, I run the below command [postgres@CPPMoma-DB05 bin]$ pg_dump -U postgres moma_ts_oce; echo $ Output was like this(Last few lines ) ------------------------------ -- Name: public; Type: ACL; Schema: -; Owner: postgres -- REVOKE ALL ON SCHEMA public FROM PUBLIC; REVOKE ALL ON SCHEMA public FROM postgres; GRANT ALL ON SCHEMA public TO postgres; GRANT ALL ON SCHEMA public TO PUBLIC; -- -- PostgreSQL database dump complete -- $ Regards, Dinesh Chandra |Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida. ------------------------------------------------------------------ Mobile: +91-9953975849 | Ext 1078 |dinesh.chandra@cyient.com Plot No. 7, NSEZ, Phase-II ,Noida-Dadri Road, Noida - 201 305,India. -----Original Message----- From: vinny [mailto:vinny@xs4all.nl] Sent: 27 February, 2017 7:31 PM To: John Gorman <jgorman@eldocomp.com> Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org;pgsql-performance-owner@postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ? On 2017-02-27 14:29, John Gorman wrote: > Even though it's not listed in any of the documentation or “pg_dump > --help” you can check the return code of the process. A return code > greater than 0 (zero) usually indicates a failure > > ./bin >pg_dump -U dummy_user dummy_database; echo $? > > 1 > > FROM: pgsql-performance-owner@postgresql.org > [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick > Otten > SENT: Monday, February 27, 2017 3:36 AM > TO: Dinesh Chandra 12108 > CC: pgsql-performance@postgresql.org > SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is > successfully or not ? > > Although it doesn't really tell if the pg_dump was successful (you'll > need to do a full restore to be sure), I generate an archive list. If > that fails, the backup clearly wasn't successful, and if it succeeds, > odds are pretty good that it worked: > > On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 > <Dinesh.Chandra@cyient.com> wrote: > > Hi, > > We are taking daily full backup of PostgreSQL database using PG_DUMP > which is automatic scheduled through Cronjobs. > > How can I check my yesterday backup is successfully or not? > > Is there any query or view by which I can check it? > > REGARDS, > > DINESH CHANDRA > > |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA. It's important to note the distinction between "the backup process did not fail" and "we now have a trustworthy backup" And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database. So what is it that you want to make sure of: 1. Did the process give an error? 2. Did the process create a usable backup? What are the chances of #1 reporting success but still producing a bad backup? And can #2 fail on a good database, and if so, can you detect that? ________________________________ DISCLAIMER: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information.Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient,please contact the sender by reply email and destroy all copies of the original message. Check all attachmentsfor viruses before opening them. All views or opinions presented in this e-mail are those of the author and maynot reflect the opinion of Cyient or those of our affiliates.
Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?
Hi, When I issue the bleow command > ./bin >pg_dump -U dummy_user dummy_database; echo $? I checked with Linux TOP command on the same server, it was showing COPY database. What exactly it doing ?? Regards, Dinesh Chandra -----Original Message----- From: vinny [mailto:vinny@xs4all.nl] Sent: 27 February, 2017 7:31 PM To: John Gorman <jgorman@eldocomp.com> Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org;pgsql-performance-owner@postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ? On 2017-02-27 14:29, John Gorman wrote: > Even though it's not listed in any of the documentation or “pg_dump > --help” you can check the return code of the process. A return code > greater than 0 (zero) usually indicates a failure > > ./bin >pg_dump -U dummy_user dummy_database; echo $? > > 1 > > FROM: pgsql-performance-owner@postgresql.org > [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick > Otten > SENT: Monday, February 27, 2017 3:36 AM > TO: Dinesh Chandra 12108 > CC: pgsql-performance@postgresql.org > SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is > successfully or not ? > > Although it doesn't really tell if the pg_dump was successful (you'll > need to do a full restore to be sure), I generate an archive list. If > that fails, the backup clearly wasn't successful, and if it succeeds, > odds are pretty good that it worked: > > On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108 > <Dinesh.Chandra@cyient.com> wrote: > > Hi, > > We are taking daily full backup of PostgreSQL database using PG_DUMP > which is automatic scheduled through Cronjobs. > > How can I check my yesterday backup is successfully or not? > > Is there any query or view by which I can check it? > > REGARDS, > > DINESH CHANDRA > > |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA. It's important to note the distinction between "the backup process did not fail" and "we now have a trustworthy backup" And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database. So what is it that you want to make sure of: 1. Did the process give an error? 2. Did the process create a usable backup? What are the chances of #1 reporting success but still producing a bad backup? And can #2 fail on a good database, and if so, can you detect that? ________________________________ DISCLAIMER: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information.Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient,please contact the sender by reply email and destroy all copies of the original message. Check all attachmentsfor viruses before opening them. All views or opinions presented in this e-mail are those of the author and maynot reflect the opinion of Cyient or those of our affiliates.
Hi,
When I issue the bleow command
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??
Regards,
Dinesh Chandra
-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@ postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list. If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.
It's important to note the distinction between
"the backup process did not fail"
and
"we now have a trustworthy backup"
And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.
So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?
What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?--________________________________
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?
Dear Rick,
Thanks for your valuable reply.
But the daily restoration of backup to another database is really so time consuming because our databases size is greater than 2TB.
Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)|
From: Rick Otten [mailto:rottenwindfish@gmail.com]
Sent: 02 March, 2017 6:45 PM
To: Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk. By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.
The best way to ensure good backups is to establish a regular practice of restoring a backup to another database. The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis. Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.
pg_dump uses the ordinary postgresql COPY command to extract data from the tables. Beyond that, I'm not sure how it works. Sorry I can't help you there.
On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,
When I issue the bleow command
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??
Regards,
Dinesh Chandra
-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.orgSubject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list. If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.
It's important to note the distinction between
"the backup process did not fail"
and
"we now have a trustworthy backup"
And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.
So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?
What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?________________________________
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?
From: Rick Otten Sent: Thursday, March 2, 2017 08:19 To: Dinesh Chandra 12108 Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ? |
Hi,
When I issue the bleow command
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??
Regards,
Dinesh Chandra
-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@ postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list. If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.
It's important to note the distinction between
"the backup process did not fail"
and
"we now have a trustworthy backup"
And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.
So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?
What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?--________________________________
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] How Can I check PostgreSQL backup is successfully ornot ?
May you please share what types of check if there to guarantee the backup is trustworthy.
Regards,
Dinesh Chandra
|Database administrator (Oracle/PostgreSQL)| Cyient Ltd. Noida.
From: ldh@laurent-hasson.com [mailto:ldh@laurent-hasson.com]
Sent: 02 March, 2017 9:57 PM
To: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
It'd be so nice to have some checks to guarantee the backup is trustworthy. Restoring the db is imho not a very good option in general:
- large databases are a problem. My db is about 3TB. Time plus disk space is a big blocker.
- also, what if the backup is incomplete? Just restoring the db successfully is not enough right? You'd have to compare with the prod to make sure nothing was missed... in a fast moving outfit where the db today will have tons of new/changed deleted stuff from yesterday.. how to even do that?
I am in a warehouse environment, so I have given up on guaranteeing backups and in a case of trouble, i'll spend 20h rebuilding my db. So I have a way out but i'd much prefer working with trustworthy backups.
Sent from my BlackBerry 10 smartphone.
From: Rick Otten Sent: Thursday, March 2, 2017 08:19 To: Dinesh Chandra 12108 Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ? |
This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk. By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.
The best way to ensure good backups is to establish a regular practice of restoring a backup to another database. The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis. Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.
pg_dump uses the ordinary postgresql COPY command to extract data from the tables. Beyond that, I'm not sure how it works. Sorry I can't help you there.
On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:
Hi,
When I issue the bleow command
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??
Regards,
Dinesh Chandra
-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.orgSubject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list. If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.
It's important to note the distinction between
"the backup process did not fail"
and
"we now have a trustworthy backup"
And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.
So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?
What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?________________________________
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
On 2 Mar 2017, at 13:26, ldh@laurent-hasson.com wrote:It'd be so nice to have some checks to guarantee the backup is trustworthy. Restoring the db is imho not a very good option in general:- large databases are a problem. My db is about 3TB. Time plus disk space is a big blocker.- also, what if the backup is incomplete? Just restoring the db successfully is not enough right? You'd have to compare with the prod to make sure nothing was missed... in a fast moving outfit where the db today will have tons of new/changed deleted stuff from yesterday.. how to even do that?I am in a warehouse environment, so I have given up on guaranteeing backups and in a case of trouble, i'll spend 20h rebuilding my db. So I have a way out but i'd much prefer working with trustworthy backups.Sent from my BlackBerry 10 smartphone.
From: Rick OttenSent: Thursday, March 2, 2017 08:19To: Dinesh Chandra 12108Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?This reminds me - I have had a case where the exit code for pg_dump was successful, but the backup was still corrupted on disk. By all means check the exit code, but I strong encourage a second validation, such as the index listing, to increase your confidence that the backup was successful.The best way to ensure good backups is to establish a regular practice of restoring a backup to another database. The easiest such practice to justify and implement is to maintain a developer/development database, and to use your production database backups to rebuild it on a regular basis. Other approaches could include regularly scheduled Disaster Recovery exercises, or simply spinning up throw away cloud instances for the purpose.pg_dump uses the ordinary postgresql COPY command to extract data from the tables. Beyond that, I'm not sure how it works. Sorry I can't help you there.On Thu, Mar 2, 2017 at 7:05 AM, Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com> wrote:Hi,
When I issue the bleow command
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
I checked with Linux TOP command on the same server, it was showing COPY database.
What exactly it doing ??
Regards,
Dinesh Chandra
-----Original Message-----
From: vinny [mailto:vinny@xs4all.nl]
Sent: 27 February, 2017 7:31 PM
To: John Gorman <jgorman@eldocomp.com>
Cc: Rick Otten <rottenwindfish@gmail.com>; Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>; pgsql-performance@postgresql.org; pgsql-performance-owner@ postgresql.org Subject: Re: [PERFORM] How Can I check PostgreSQL backup is successfully or not ?
On 2017-02-27 14:29, John Gorman wrote:
> Even though it's not listed in any of the documentation or “pg_dump
> --help” you can check the return code of the process. A return code
> greater than 0 (zero) usually indicates a failure
>
> ./bin >pg_dump -U dummy_user dummy_database; echo $?
>
> 1
>
> FROM: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] ON BEHALF OF Rick
> Otten
> SENT: Monday, February 27, 2017 3:36 AM
> TO: Dinesh Chandra 12108
> CC: pgsql-performance@postgresql.org
> SUBJECT: Re: [PERFORM] How Can I check PostgreSQL backup is
> successfully or not ?
>
> Although it doesn't really tell if the pg_dump was successful (you'll
> need to do a full restore to be sure), I generate an archive list. If
> that fails, the backup clearly wasn't successful, and if it succeeds,
> odds are pretty good that it worked:
>
> On Mon, Feb 27, 2017 at 4:35 AM, Dinesh Chandra 12108
> <Dinesh.Chandra@cyient.com> wrote:
>
> Hi,
>
> We are taking daily full backup of PostgreSQL database using PG_DUMP
> which is automatic scheduled through Cronjobs.
>
> How can I check my yesterday backup is successfully or not?
>
> Is there any query or view by which I can check it?
>
> REGARDS,
>
> DINESH CHANDRA
>
> |DATABASE ADMINISTRATOR (ORACLE/POSTGRESQL)| CYIENT LTD. NOIDA.
It's important to note the distinction between
"the backup process did not fail"
and
"we now have a trustworthy backup"
And you can go full-paranoia and say that you can successfully create a perfectly working backup of the wrong database.
So what is it that you want to make sure of:
1. Did the process give an error?
2. Did the process create a usable backup?
What are the chances of #1 reporting success but still producing a bad backup?
And can #2 fail on a good database, and if so, can you detect that?--________________________________
DISCLAIMER:
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Check all attachments for viruses before opening them. All views or opinions presented in this e-mail are those of the author and may not reflect the opinion of Cyient or those of our affiliates.
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance