Thread: Monitoring database for changes - backup purposes

Monitoring database for changes - backup purposes

From
Chris Jewell
Date:
Hi,

I have a query about a backup policy that I wish to implement.
Currently, we are running a PostgreSQL server with a very large database
on it.  The database consists of a core of read-only tables, with tables
created by the users going by default into the user's own schemas.  For
backup purposes, I have a script that uses pg_dump to create dumps of
the individual user schemas.  However, this is mainly a read-only
database and information is added or updated only rarely.  What I really
need is a method of monitoring the database, such that my backup script
will only back any one schema up if it has changed since it was last
backed up.  Is there any way to do this?  Is there a checksum function
or something that I could use?

Thanks,

Chris

--
Chris Jewell, BSc(Hons), BVSc, MRCVS
Dept of Maths and Statistics
Fylde College
Lancaster University
Lancaster
Lancs
LA1 4YF


Re: Monitoring database for changes - backup purposes

From
Darcy Buskermolen
Date:
On Friday 14 October 2005 06:19, Chris Jewell wrote:
> Hi,
>
> I have a query about a backup policy that I wish to implement.
> Currently, we are running a PostgreSQL server with a very large database
> on it.  The database consists of a core of read-only tables, with tables
> created by the users going by default into the user's own schemas.  For
> backup purposes, I have a script that uses pg_dump to create dumps of
> the individual user schemas.  However, this is mainly a read-only
> database and information is added or updated only rarely.  What I really
> need is a method of monitoring the database, such that my backup script
> will only back any one schema up if it has changed since it was last
> backed up.  Is there any way to do this?  Is there a checksum function
> or something that I could use?

No there is not, but would a diff -q be sufficient for your needs ?


>
> Thanks,
>
> Chris

--
Darcy Buskermolen
Wavefire Technologies Corp.

http://www.wavefire.com
ph: 250.717.0200
fx: 250.763.1759

Re: Monitoring database for changes - backup purposes

From
Chris Jewell
Date:
Hi,

Yes, diff would work and is my current plan, but it requires me to
perform the backup first.  What I needed really was a system by which a
backup would only be run if there was something that needed backing up.

Thanks anyway,

Chris

--
Chris Jewell, BSc(Hons), BVSc, MRCVS
Dept of Maths and Statistics
Fylde College
Lancaster University
Lancaster
Lancs
LA1 4YF



Darcy Buskermolen wrote:

>On Friday 14 October 2005 06:19, Chris Jewell wrote:
>
>
>>Hi,
>>
>>I have a query about a backup policy that I wish to implement.
>>Currently, we are running a PostgreSQL server with a very large database
>>on it.  The database consists of a core of read-only tables, with tables
>>created by the users going by default into the user's own schemas.  For
>>backup purposes, I have a script that uses pg_dump to create dumps of
>>the individual user schemas.  However, this is mainly a read-only
>>database and information is added or updated only rarely.  What I really
>>need is a method of monitoring the database, such that my backup script
>>will only back any one schema up if it has changed since it was last
>>backed up.  Is there any way to do this?  Is there a checksum function
>>or something that I could use?
>>
>>
>
>No there is not, but would a diff -q be sufficient for your needs ?
>
>
>
>
>>Thanks,
>>
>>Chris
>>
>>
>
>
>

Re: Monitoring database for changes - backup purposes

From
"Jim C. Nasby"
Date:
On Fri, Oct 14, 2005 at 07:29:31PM +0100, Chris Jewell wrote:
> Hi,
>
> Yes, diff would work and is my current plan, but it requires me to
> perform the backup first.  What I needed really was a system by which a
> backup would only be run if there was something that needed backing up.

I believe he meant diffing the files that store the tables.

Another possibility that I think would work would be checking the
last-modified timestamp of all the files in a schema. Though the easiest
way to do that is probably to build a list of any files that have
changed since the last backup and then figure out what schemas they're
in.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

Re: Monitoring database for changes - backup purposes

From
sualeh.fatehi@gmail.com
Date:
Chris,

If you have a JDBC driver for pgsql, you can use SchemaCrawler.
https://sourceforge.net/project/showfiles.php?group_id=148383

SchemaCrawler is a free, open source tool that can produce database
schema metadata and/ or data output in a diff-able format.

Sualeh.


Chris Jewell wrote:
> Hi,
>
> Yes, diff would work and is my current plan, but it requires me to
> perform the backup first.  What I needed really was a system by which a
> backup would only be run if there was something that needed backing up.
>
> Thanks anyway,
>
> Chris
>
> --
> Chris Jewell, BSc(Hons), BVSc, MRCVS
> Dept of Maths and Statistics
> Fylde College
> Lancaster University
> Lancaster
> Lancs
> LA1 4YF
>
>
>
> Darcy Buskermolen wrote:
>
> >On Friday 14 October 2005 06:19, Chris Jewell wrote:
> >
> >
> >>Hi,
> >>
> >>I have a query about a backup policy that I wish to implement.
> >>Currently, we are running a PostgreSQL server with a very large database
> >>on it.  The database consists of a core of read-only tables, with tables
> >>created by the users going by default into the user's own schemas.  For
> >>backup purposes, I have a script that uses pg_dump to create dumps of
> >>the individual user schemas.  However, this is mainly a read-only
> >>database and information is added or updated only rarely.  What I really
> >>need is a method of monitoring the database, such that my backup script
> >>will only back any one schema up if it has changed since it was last
> >>backed up.  Is there any way to do this?  Is there a checksum function
> >>or something that I could use?
> >>
> >>
> >
> >No there is not, but would a diff -q be sufficient for your needs ?
> >
> >
> >
> >
> >>Thanks,
> >>
> >>Chris
> >>
> >>
> >
> >
> >
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq