Re: Implementing incremental backup - Mailing list pgsql-hackers

From Tatsuo Ishii
Subject Re: Implementing incremental backup
Date
Msg-id 20130620.084042.49354019011506078.t-ishii@sraoss.co.jp
Whole thread Raw
In response to Re: Implementing incremental backup  (Claudio Freire <klaussfreire@gmail.com>)
Responses Re: Implementing incremental backup
Re: Implementing incremental backup
List pgsql-hackers
> On Wed, Jun 19, 2013 at 6:20 PM, Stephen Frost <sfrost@snowman.net> wrote:
>> * Claudio Freire (klaussfreire@gmail.com) wrote:
>>> I don't see how this is better than snapshotting at the filesystem
>>> level. I have no experience with TB scale databases (I've been limited
>>> to only hundreds of GB), but from my limited mid-size db experience,
>>> filesystem snapshotting is pretty much the same thing you propose
>>> there (xfs_freeze), and it works pretty well. There's even automated
>>> tools to do that, like bacula, and they can handle incremental
>>> snapshots.
>>
>> Large databases tend to have multiple filesystems and getting a single,
>> consistent, snapshot across all of them while under load is..
>> 'challenging'.  It's fine if you use pg_start/stop_backup() and you're
>> saving the XLOGs off, but if you can't do that..
> 
> Good point there.
> 
> I still don't like the idea of having to mark each modified page. The
> WAL compressor idea sounds a lot more workable. As in scalable.

Why do you think WAL compressor idea is more scalable? I really want
to know why. Besides the unlogged tables issue, I can accept the idea
if WAL based solution is much more efficient. If there's no perfect,
ideal solution, we need to prioritize things. My #1 priority is
allowing to create incremental backup against TB database, and the
backup file should be small enough and the time to create it is
acceptable. I just don't know why scanning WAL stream is much cheaper
than recording modified page information.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp



pgsql-hackers by date:

Previous
From: Tatsuo Ishii
Date:
Subject: Re: Implementing incremental backup
Next
From: Stephen Frost
Date:
Subject: Re: Implementing incremental backup