Re: block-level incremental backup - Mailing list pgsql-hackers

From Jehan-Guillaume de Rorthais
Subject Re: block-level incremental backup
Date
Msg-id 20190410224603.3b63ee74@firost
Whole thread Raw
In response to Re: block-level incremental backup  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Wed, 10 Apr 2019 14:38:43 -0400
Robert Haas <robertmhaas@gmail.com> wrote:

> On Wed, Apr 10, 2019 at 2:21 PM Jehan-Guillaume de Rorthais
> <jgdr@dalibo.com> wrote:
> > In my current design, the scan is done backward from end to start and I
> > keep all the records appearing after the last occurrence of their
> > respective FPI.  
> 
> Oh, interesting.  That seems like it would require pretty major
> surgery on the WAL stream.

Indeed.

Presently, the surgery in my code is replacing redundant xlogrecord with noop.

I have now to deal with muti-blocks records. So far, I tried to mark non-needed
block with !BKPBLOCK_HAS_DATA and made a simple patch in core to ignore such
marked blocks, but it doesn't play well with dependency between xlogrecord, eg.
during UPDATE. So my plan is to rewrite them to remove non-needed blocks using
eg. XLOG_FPI.

As I wrote, this is mainly an hobby project right now for my own education. Not
sure where it leads me, but I learn a lot while working on it.



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: pg_dump is broken for partition tablespaces
Next
From: Jehan-Guillaume de Rorthais
Date:
Subject: Re: block-level incremental backup