Re: Proposal: Incremental Backup - Mailing list pgsql-hackers

<div dir="ltr"><br /><div class="gmail_extra"><br /><br /><div class="gmail_quote">2014-08-01 18:20 GMT+02:00 Claudio
Freire<span dir="ltr"><<a href="mailto:klaussfreire@gmail.com"
target="_blank">klaussfreire@gmail.com</a>></span>:<br/><blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px#ccc solid;padding-left:1ex"><div class="">On Fri, Aug 1, 2014 at 12:35 AM, Amit Kapila <<a
href="mailto:amit.kapila16@gmail.com">amit.kapila16@gmail.com</a>>wrote:<br /> >> c) the map is not crash safe
bydesign, because it needs only for<br /> >> incremental backup to track what blocks needs to be backuped, not
for<br/> >> consistency or recovery of the whole cluster, so it's not an heavy cost for<br /> >> the whole
clusterto maintain it. we could think an option (but it's heavy)<br /> >> to write it at every flush  on file to
havecrash-safe map, but I not think<br /> >> it's so usefull . I think it's acceptable, and probably it's better
toforce<br /> >> that, to say: "if your db will crash, you need a fullbackup ",<br /> ><br /> > I am not
sureif your this assumption is right/acceptable, how can<br /> > we say that in such a case users will be okay to
havea fullbackup?<br /> > In general, taking fullbackup is very heavy operation and we should<br /> > try to
avoidsuch a situation.<br /><br /><br /></div>Besides, the one taking the backup (ie: script) may not be aware of<br />
theneed to take a full one.<br /><br /> It's a bad design to allow broken backups at all, IMNSHO.<br
/></blockquote></div><br/></div><div class="gmail_extra">Hi Claudio, <br /></div><div
class="gmail_extra">                thanks for your observation<br /></div><div class="gmail_extra">First: the case
it'safter a crash of a database, and it's not something happens every day or every week. It's something that happens in
rareconditions, or almost my experience is so. If it happens very often probably there are other problems.<br
/></div><divclass="gmail_extra">Second: to avoid the problem to know if the db needed to have a full backup to rebuild
themap we could think to write in the map header the backup reference (with an id and LSN reference for example ) so 
ifthe someone/something try to do an incremental backup after a crash, the map header will not have noone full backup
listed[because it will be empty] , and automaticcaly switch to a full one. I think after a crash it's a good practice
todo a full backup, to see if there are some problems on files or on filesystems, but if I am wrong I am happy to know
:). <br /><br /></div><div class="gmail_extra">Remember that  I propose a map in ram to reduce the impact on
performances,but we could create an option to leave the choose to the user, if you want a crash safe map, at every
flushwill be updated also a map file , if not, the map will be in ram.<br /><br /></div><div class="gmail_extra">Kind
Regards<br/><br /><br /></div><div class="gmail_extra">Mat<br /></div></div>