Re: Implementing incremental backup - Mailing list pgsql-hackers

From Claudio Freire
Subject Re: Implementing incremental backup
Date
Msg-id CAGTBQpbmwcv1tP3q4ee8KbQp_y0vmc3nNr8jYjeg3-xud+fOhw@mail.gmail.com
Whole thread Raw
In response to Re: Implementing incremental backup  (Tatsuo Ishii <ishii@postgresql.org>)
Responses Re: Implementing incremental backup  (Tatsuo Ishii <ishii@postgresql.org>)
List pgsql-hackers
On Wed, Jun 19, 2013 at 7:39 PM, Tatsuo Ishii <ishii@postgresql.org> wrote:
>>> I'm thinking of implementing an incremental backup tool for
>>> PostgreSQL. The use case for the tool would be taking a backup of huge
>>> database. For that size of database, pg_dump is too slow, even WAL
>>> archive is too slow/ineffective as well. However even in a TB
>>> database, sometimes actual modified blocks are not that big, may be
>>> even several GB. So if we can backup those modified blocks only,
>>> that would be an effective incremental backup method.
>>
>> I'm trying to figure out how that's actually different from WAL..?  It
>> sounds like you'd get what you're suggesting with simply increasing the
>> checkpoint timeout until the WAL stream is something which you can keep
>> up with.  Of course, the downside there is that you'd have to replay
>> more WAL when recovering.
>
> Yeah, at first I thought using WAL was a good idea.  However I realized
> that the problem using WAL is we cannot backup unlogged tables because
> they are not written to WAL.

How does replication handle that?

Because I doubt that's an issue only with backups.



pgsql-hackers by date:

Previous
From: Tatsuo Ishii
Date:
Subject: Re: Implementing incremental backup
Next
From: Jeff Janes
Date:
Subject: Re: Change authentication error message (patch)