Thread: Using pg_rewind for differential backup

Using pg_rewind for differential backup

From
Sameer Kumar
Date:
Can we tweak pg_rewind to take differential backups in PostgreSQL?

I was wondering can we hack the pg_rewind code to print the details of the file which have been modified compared to a target server. 
The list output can then be used for taking differential backups.

Or perhaps we can add an option/switch in pg_rewind --action

--action=print ---> would print the files which have changed
--action=sync ---> would sync them
--action=copy ---> with this option I can specify an additional optino --target-dir where I can copy the files which have changed blocks


pg_rewind- 


Best Regards,

Sameer Kumar | Database Consultant

ASHNIK PTE. LTD.

101 Cecil Street, #11-11 Tong Eng Building, Singapore 069533

M: +65 8110 0350  T: +65 6438 3504 | www.ashnik.com

icons

 

Email patch

 

This email may contain confidential, privileged or copyright material and is solely for the use of the intended recipient(s).

Attachment

Re: Using pg_rewind for differential backup

From
Michael Paquier
Date:


On Thu, Nov 27, 2014 at 9:39 PM, Sameer Kumar <sameer.kumar@ashnik.com> wrote:
Can we tweak pg_rewind to take differential backups in PostgreSQL?
I was wondering can we hack the pg_rewind code to print the details of the file which have been modified compared to a target server. The list output can then be used for taking differential backups.

Or perhaps we can add an option/switch in pg_rewind --action

--action=print ---> would print the files which have changed
--action=sync ---> would sync them
--action=copy ---> with this option I can specify an additional optino --target-dir where I can copy the files which have changed

This discussion is not really adapted on hackers as pg_rewind is not included in Postgres core code. Please let's discuss your proposal there. Btw, pg_rewind is not aimed to be used as a tool for a backup facility. You may find easier to use existing backup solutions instead, or help out with an in-core solution. There was actually not so long ago a patch to implement in-core differential backups, with a part called profiles able to fetch back the list of files modified since a given LSN point (only an improvement of the core feature though as that's not mandatory to make the machinery work). The patch has not been added to any of the recent commit fests though.
Regards,
--
Michael

Re: Using pg_rewind for differential backup

From
Heikki Linnakangas
Date:
On 11/28/2014 09:30 AM, Michael Paquier wrote:
> On Thu, Nov 27, 2014 at 9:39 PM, Sameer Kumar <sameer.kumar@ashnik.com>
> wrote:
>
>> Can we tweak pg_rewind to take differential backups in PostgreSQL?
>> I was wondering can we hack the pg_rewind code to print the details of the
>> file which have been modified compared to a target server. The list output
>> can then be used for taking differential backups.
>>
>> Or perhaps we can add an option/switch in pg_rewind --action
>>
>> --action=print ---> would print the files which have changed
>> --action=sync ---> would sync them
>> --action=copy ---> with this option I can specify an additional optino
>> --target-dir where I can copy the files which have changed
>>
>
> This discussion is not really adapted on hackers as pg_rewind is not
> included in Postgres core code. Please let's discuss your proposal there.
> Btw, pg_rewind is not aimed to be used as a tool for a backup facility. You
> may find easier to use existing backup solutions instead, or help out with
> an in-core solution.

It also would be quite straightforward to write a separate tool to do 
just that. Would be better than conflating pg_rewind with this. You 
could use pg_rewind as the basis for it - it's under the same license as 
PostgreSQL.

- Heikki




Re: Using pg_rewind for differential backup

From
Robert Haas
Date:
On Fri, Nov 28, 2014 at 2:49 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> It also would be quite straightforward to write a separate tool to do just
> that. Would be better than conflating pg_rewind with this. You could use
> pg_rewind as the basis for it - it's under the same license as PostgreSQL.

If we had such a tool in core, would that completely solve the
differential backup problem, or would more be needed?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Using pg_rewind for differential backup

From
Sameer Kumar
Date:

On Wed, Dec 3, 2014 at 1:32 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Nov 28, 2014 at 2:49 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> It also would be quite straightforward to write a separate tool to do just
> that. Would be better than conflating pg_rewind with this. You could use
> pg_rewind as the basis for it - it's under the same license as PostgreSQL.

​Agree. I would want to work on that.​
 
If we had such a tool in core, would that completely solve the
differential backup problem, or would more be needed?

​In my opinion​ yes. 

This discussion is not really adapted on hackers as pg_rewind is not included in Postgres core code. Please let's discuss your proposal there.

​Got it​. But given the above, can we keep this discussion thread here?



Best Regards,

Sameer Kumar | Database Consultant

ASHNIK PTE. LTD.

101 Cecil Street, #11-11 Tong Eng Building, Singapore 069533

M: +65 8110 0350  T: +65 6438 3504 | www.ashnik.com

icons

 

Email patch

 

This email may contain confidential, privileged or copyright material and is solely for the use of the intended recipient(s).

Attachment