Re: Backing up 16TB of data (was Re: > 16TB worth of - Mailing list pgsql-general

From Fred Moyer
Subject Re: Backing up 16TB of data (was Re: > 16TB worth of
Date
Msg-id 33237.68.73.117.46.1051456532.squirrel@mail.digicamp.com
Whole thread Raw
In response to Re: Backing up 16TB of data (was Re: > 16TB worth of  (Ron Johnson <ron.l.johnson@cox.net>)
List pgsql-general
Here's a 70 TB Backup RAID at the University of Tübingen that should be
able handle the needs adequately.

http://www.tomshardware.com/storage/20030425/index.html

> On Sat, 2003-04-26 at 19:18, Jan Wieck wrote:
>> Lincoln Yeoh wrote:
>> >
>> > At 07:32 PM 4/25/2003 -0400, Jan Wieck wrote:
>> > >Of course, assuming we want to backup the total 24 Terabyte he has
>> in 2-3 years in less than a day, if we have a month to take a
>> backup we can save some money on the backup solution.
>> >
>> > If you take the month approach, the filesystem/db snapshot has to
>> stay in place for a full month whilst the backup is occuring.
>> Do-able, but can be stressful e.g. if something goes wrong after
>> trying for a full month...
>> >
>> > But anyway he said off-line backups aren't that important - I gather
>> that recreating the data is not totally impractical. Still 16TB,
>> ouch.
>>
>> I think a scenario like that, where one has a relatively small
>> percentage of really updated data and a huge portion of constantly
>> growing, is a good example for when it might be appropriate NOT to
>> store everything in one database.
>>
>> Depending on the rest of the requirements, it might need 2PC and using
>> 2 databases. The updated database then could be backed up by normal
>> means while for the constantly growing one you just archive the redo
>> logs.
>
> Here's another thought: does all 16TB of data *really* have to be
> in the database all the time??
>
> Maybe there's a business rule that anything older than 6 months isn't in
> the database itself, but is "just" sitting out on somewhere on a
> filesystem, and if the old data is requested, then, either
> programmatically or thru operator intervention, the old data is copied
> into the database.
>
> Yes, it would take longer to access "old" data, but, hey, that's
> reality (unless you want to spend *really*large* amounts of money). And
> it's not an "OSS vs.Proprietary" either.


pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: Resetting a sequence's last value...
Next
From: Tom Lane
Date:
Subject: Re: Cannot use the queries buffer of psql... :-(