Thread: Shell Script help for backup
Hey all,
I'm thinkin' maybe I should post this in a shell scripting group but
thought someone here may be able to help me out...
I'm lookin' to run a shell script through the crontab on a nightly
basis for backup, and not keep more than a week of backups. In those
terms, I have a script working.
But I wanna take the script to another level. Basically, I want it to
email me if pg_dump fails, but I'm having a hard time doing so.
here's what I got
#!/bin/bash
DATE=`date +%Y-%m-%d`
PGUSER=<my username>
PGPASSWORD=<my password>
export PGUSER PGPASSWORD
pg_dump <dbname> | gzip > /folder/pg_backup.$DATE.gzip
if [ !$? ]
then
find /folder/pg_backup/* -mtime 8 -exec rm -f {} \;
echo "The PostGreSQL backup for today completed successfully" | mail
ratlhead@ratlhead.com
else
echo "The PostGreSQL backup for today was unsuccessful" | mail
ratlhead@ratlhead.com
fi
The problem is that is always gets detected as being successful, even
if I say, comment out the export line of my user/pass. Any
suggestions on how I can detect whether or not it worked?
Something else I'll be lookin' to do is FTP'ing the backup to another
server. If anyone has suggestions on where I can look for help on
that, it'd be great.
Thanks!
On Sat, Jul 20, 2002 at 02:25:48AM -0700, ratlhead wrote:
>
> #!/bin/bash
> DATE=`date +%Y-%m-%d`
> PGUSER=<my username>
> PGPASSWORD=<my password>
> export PGUSER PGPASSWORD
> pg_dump <dbname> | gzip > /folder/pg_backup.$DATE.gzip
> if [ !$? ]
> then
> find /folder/pg_backup/* -mtime 8 -exec rm -f {} \;
> echo "The PostGreSQL backup for today completed successfully" | mail
> ratlhead@ratlhead.com
> else
> echo "The PostGreSQL backup for today was unsuccessful" | mail
> ratlhead@ratlhead.com
> fi
>
>
> The problem is that is always gets detected as being successful, even
> if I say, comment out the export line of my user/pass. Any
> suggestions on how I can detect whether or not it worked?
The script always detects a successful dump because the last command in
the pipe is gzip and it always succeeds.
You could change the [ !$? ] to something like:
[ "$( zcat /folder/pg_backup.$DATE.gzip | head -c1 | wc -c )" -eq 1 ]
> Something else I'll be lookin' to do is FTP'ing the backup to another
> server. If anyone has suggestions on where I can look for help on
> that, it'd be great.
Use ncftpput or scp with no passphrase.
Regards,
Luciano Rocha
--
Consciousness: that annoying time between naps.
You don't have to compromise and use a clear passphrase, you can use keychain, this uses ssh-agent and caches the user's passphrases: http://www-106.ibm.com/developerworks/library/l-keyc2/ -----Original Message----- From: strange@nsk.yi.org [mailto:strange@nsk.yi.org] Sent: 22 July 2002 15:09 To: pgsql-general@postgresql.org Subject: Re: [GENERAL] Shell Script help for backup On Sat, Jul 20, 2002 at 02:25:48AM -0700, ratlhead wrote: > > #!/bin/bash > DATE=`date +%Y-%m-%d` > PGUSER=<my username> > PGPASSWORD=<my password> > export PGUSER PGPASSWORD > pg_dump <dbname> | gzip > /folder/pg_backup.$DATE.gzip > if [ !$? ] > then > find /folder/pg_backup/* -mtime 8 -exec rm -f {} \; > echo "The PostGreSQL backup for today completed successfully" | mail > ratlhead@ratlhead.com > else > echo "The PostGreSQL backup for today was unsuccessful" | mail > ratlhead@ratlhead.com > fi > > > The problem is that is always gets detected as being successful, even > if I say, comment out the export line of my user/pass. Any > suggestions on how I can detect whether or not it worked? The script always detects a successful dump because the last command in the pipe is gzip and it always succeeds. You could change the [ !$? ] to something like: [ "$( zcat /folder/pg_backup.$DATE.gzip | head -c1 | wc -c )" -eq 1 ] > Something else I'll be lookin' to do is FTP'ing the backup to another > server. If anyone has suggestions on where I can look for help on > that, it'd be great. Use ncftpput or scp with no passphrase. Regards, Luciano Rocha -- Consciousness: that annoying time between naps. ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org Mae�r neges e-bost hon ac unrhyw ffeiliau sydd ynghlwm wrthi yn gwbl gyfrinachol ac wedi�u bwriadu at sylw yr unigolyn neu�rendid y maent wedi eu cyfeirio ato yn unig. Gallant gynnwys gwybodaeth sy�n freintiedig yn gyfreithiol a/neu�n gyfrinacholac os nad chi yw�r sawl y cyfeiriwyd y neges ato, yna peidiwch � chopio, dosbarthu na chymryd unrhyw gamau o ganlyniada gan ddibynnu arni. Os ydych wedi derbyn y neges e-bost hon ar gam, a wnewch chi ein hysbysu o hyn cyn gynted �phosib a�i dileu os gwelwch yn dda. Barn neu safbwyntiau�r awdur yw�r rhai a fynegir yn y neges e-bost hon ac/neu�n unrhywatodiadau ac nid ydynt yn adlewyrchu o anghenraid barn neu safbwyntiau S4C. This e-mail message and any files attached are strictly confidential and intended solely for the attention of the personor entity to whom they are addressed. They may contain legally privileged and/or confidential information and if youare not the addressee, you should not copy, distribute nor take any action in reliance on them. If you have received thise-mail message in error, please notify us as soon as possible and delete it. The views and opinions expressed in thise-mail message and any attachments are the author�s own and they do not reflect necessarily the views and opinions ofS4C.
Hi all,
This is how I ended up solving my problem. Not sure if any of this
will help you, but here's the scripts I use. This is what it does...
-Uses PG environmental variables to connect to the PG server
-Dumps the specified database
-Checks the backup folder for any backups older than a week and
deletes those
-Checks to be sure the backup was successful by checking for a 0k file
for that day
-Sends email saying if backup was successful or failed
-If successful, checks the day and if it's Saturday, uploads most
current backup to specified ftp site
-.netrc file required for FTP connection
##### pg_backup.sh #####
#!/bin/bash
DATE=`date +%Y-%m-%d`
PGUSER=<my user>
PGPASSWORD=<my pass>
export PGUSER PGPASSWORD
/path/pg_dump <db name> | gzip > /path/pg_backup.$DATE.gzip
find /path/pg_backup/* -mtime 7 -exec rm -f {} \;
if test -s /path/pg_backup/pg_backup.$DATE.gzip
then
echo "The PostGreSQL backup for today was successful!" | mailx -s
"PostGreSQ
L backup successful" <email address>
/path/day_pg.sh
else
echo "The PostGreSQL backup for today failed!" | mail -s
"PostGreSQL backup
failed" <email address>
fi
##### day_pg.sh #####
DAY=`date +%a`
if [ $DAY = "Sat" ]
then
/path/upload_pg.sh
fi
##### upload_pg #####
DATE=`date +%Y-%m-%d`
ftp <host> << EOF
cd path
put /path/pg_backup.$DATE.gzip pg_backup.$DATE.gzip
quit
EOF
##### .netrc #####
machine <host> login <username> password <password>
-Mike Graben
ratlhead@ratlhead.com
On Mon, Jul 22, 2002 at 03:35:01PM -0700, ratlhead wrote: > Hi all, > > This is how I ended up solving my problem. Not sure if any of this > will help you, but here's the scripts I use. This is what it does... > > -Uses PG environmental variables to connect to the PG server > -Dumps the specified database > -Checks the backup folder for any backups older than a week and > deletes those > -Checks to be sure the backup was successful by checking for a 0k file > for that day What's wrong with the solution I sent? And I doubt the checking for a 0k file will work for you as gzip will create a 20 byte file even for a 0 byte input (some headers, etc.). Regards, Luciano Rocha -- Consciousness: that annoying time between naps.
strange@nsk.yi.org wrote in message news:<20020723150902.A5173@nsk.yi.org>... > What's wrong with the solution I sent? > > And I doubt the checking for a 0k file will work for you as gzip will > create a 20 byte file even for a 0 byte input (some headers, etc.). > > Regards, > Luciano Rocha Luciano, Was wondering if you could provide some help with the solution you sent... [ "$( zcat /folder/pg_backup.$DATE.gzip | head -c1 | wc -c )" -eq 1 ] First, I can't seem to use zcat, because it likes to add on a ".Z" to the filename for some reason or another. Also, I find with the version of 'head' installed on my machine, that -c isn't available, but instead, -n. I tried changing that and zcat to cat, but was getting a fail on something that should be valid. This basically checks to see if the file contains data right? Thanks man!
You can do 'zcat < file' to prevent the *.Z requirement. Yes, head -n should work fine. --------------------------------------------------------------------------- ratlhead wrote: > strange@nsk.yi.org wrote in message news:<20020723150902.A5173@nsk.yi.org>... > > What's wrong with the solution I sent? > > > > And I doubt the checking for a 0k file will work for you as gzip will > > create a 20 byte file even for a 0 byte input (some headers, etc.). > > > > Regards, > > Luciano Rocha > > Luciano, > > Was wondering if you could provide some help with the solution you > sent... > > [ "$( zcat /folder/pg_backup.$DATE.gzip | head -c1 | wc -c )" -eq 1 ] > > First, I can't seem to use zcat, because it likes to add on a ".Z" to > the filename for some reason or another. Also, I find with the > version of 'head' installed on my machine, that -c isn't available, > but instead, -n. I tried changing that and zcat to cat, but was > getting a fail on something that should be valid. > > This basically checks to see if the file contains data right? > > Thanks man! > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
On Sun, Jul 28, 2002 at 11:22:22PM -0400, Bruce Momjian wrote: > > You can do 'zcat < file' to prevent the *.Z requirement. Yes, head -n > should work fine. Hm... I missed the other email, so I'm replying to this: The "$( zcat /folder/pg_backup.$DATE.gzip | head -c1 | wc -c )" is to check if there are any bytes stored in the gzip file. If the zcat expects an .Z file, try with gzcat file, gzcat < file, or gzip -d < file. head -c1 is meant to stop the decompression at the first byte, I didn't want it to wait until full decompression of the gzipped file, and I didn't use -n because then it would depend on the internal format of the decompressed file (if it was a binary file, a newline could appear only after quite some time). If you're going to use head -n1, then change the test to: [ "$( gzip -d < /folder/pg_backup.$DATE.gzip | head -n1 | wc -c )" -ge 1 ] (ie, instead of checking if bytes == 1, check if bytes >= 1). Regards, Luciano Rocha -- Consciousness: that annoying time between naps.