File based Incremental backup v8 - Mailing list pgsql-hackers

From Marco Nenciarini
Subject File based Incremental backup v8
Date
Msg-id 54CA481E.3050405@2ndquadrant.it
Whole thread Raw
In response to File based Incremental backup v7  (Marco Nenciarini <marco.nenciarini@2ndquadrant.it>)
Responses Re: File based Incremental backup v8
List pgsql-hackers
The current implementation of copydir function is incompatible with LSN
based incremental backups. The problem is that new files are created,
but their blocks are still with the old LSN, so they will not be backed
up because they are looking old enough.

copydir function is used in:

  CREATE DATABASE
  ALTER DATABASE SET TABLESPACE

I can imagine two possible solutions:

a) wal log the whole copydir operations, setting the lsn accordingly
b) pass to copydir the LSN of the operation which triggered it, and
update the LSN of all the copied blocks

The latter solution is IMO easier to be implemented and does not deviate
much from the current implementation.

I've implemented it and it's attached to this message.

I've also moved the parse_filename_for_notntemp_relation function out of
reinit.c to make it available both to copydir.c and basebackup.c.

I've also limited the LSN comparison to the only MAIN fork, because:

* LSN fork doesn't uses LSN
* VM fork update LSN only when the visibility bit is set
* INIT forks doesn't use LSN. It's only one page anyway.

Regards,
Marco

--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it

Attachment

pgsql-hackers by date:

Previous
From: Simon Riggs
Date:
Subject: Re: GetLockConflicts() and thus recovery conflicts seem pretty broken
Next
From: Bruce Momjian
Date:
Subject: Re: pg_upgrade and rsync