Re: [RFC] Incremental backup v2: add backup profile to base backup - Mailing list pgsql-hackers

From Marco Nenciarini
Subject Re: [RFC] Incremental backup v2: add backup profile to base backup
Date
Msg-id 542ECA0F.3030507@2ndquadrant.it
Whole thread Raw
In response to Re: [RFC] Incremental backup v2: add backup profile to base backup  (Heikki Linnakangas <hlinnakangas@vmware.com>)
Responses Re: [RFC] Incremental backup v2: add backup profile to base backup
Re: [RFC] Incremental backup v2: add backup profile to base backup
Re: [RFC] Incremental backup v2: add backup profile to base backup
List pgsql-hackers
Il 03/10/14 17:53, Heikki Linnakangas ha scritto:
> If we're going to need a profile file - and I'm not convinced of that -
> is there any reason to not always include it in the backup?
>

The main reason is to have a centralized list of files that need to be
present. Without a profile, you have to insert some sort of placeholder
for kipped files. Moreover, the profile allows you to quickly know the
size of the recovered backup (by simply summing the individual size).
Another use could be to 'validate' the presence of all required files in
a backup.

>> Any comment will be appreciated. In particular I'd appreciate comments
>> on correctness of relnode files detection and LSN extraction code.
>
> I didn't look at it in detail, but one future problem comes to mind:
> Once you implement the server-side code that only sends a file if its
> LSN is higher than the cutoff point that the client gave, you'll have to
> scan the whole file first, to see if there are any blocks with a higher
> LSN. At least until you find the first such block. So with a file-level
> implementation of this sort, you'll have to scan all files twice, in the
> worst case.
>

It's true. To solve this you have to keep a central maxLSN directory,
but I think it introduces more issues than it solves.

Regards,
Marco

--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it


pgsql-hackers by date:

Previous
From: Heikki Linnakangas
Date:
Subject: Re: replicating DROP commands across servers
Next
From: Heikki Linnakangas
Date:
Subject: Re: Last Commitfest patches waiting review