Re: WIP/PoC for parallel backup - Mailing list pgsql-hackers

From Asif Rehman
Subject Re: WIP/PoC for parallel backup
Date
Msg-id CADM=JejVJ7eaNBTtrPfzs7J+tT=PH+W1Vc24yxR2Ekwnn9eueg@mail.gmail.com
Whole thread Raw
In response to Re: WIP/PoC for parallel backup  (Asif Rehman <asifr.rehman@gmail.com>)
Responses Re: WIP/PoC for parallel backup  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers


On Tue, Dec 10, 2019 at 7:34 PM Asif Rehman <asifr.rehman@gmail.com> wrote:


On Thu, Nov 28, 2019 at 12:57 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 27, 2019 at 3:38 AM Jeevan Chalke
<jeevan.chalke@enterprisedb.com> wrote:
> I am still not sure why we need SEND_BACKUP_FILELIST as a separate command.
> Can't we return the file list with START_BACKUP itself?

I had the same thought, but I think it's better to keep them separate.
Somebody might want to use the SEND_BACKUP_FILELIST command for
something other than a backup (I actually think it should be called
just SEND_FILE_LIST)

Sure. Thanks for the recommendation. To keep the function names in sync, I intend to do following the 
following renamings:
- SEND_BACKUP_FILES --> SEND_FILES
- SEND_BACKUP_FILELIST -->  SEND_FILE_LIST

. Somebody might want to start a backup without
getting a file list because they're going to copy the files at the FS
level. Somebody might want to get a list of files to process after
somebody else has started the backup on another connection. Or maybe
nobody wants to do any of those things, but it doesn't seem to cost us
much of anything to split the commands, so I think we should.

+1 


I have updated the patches (v7 attached) and have taken care of all issues pointed by Jeevan, additionally
ran the pgindent on each patch. Furthermore, Command names have been renamed as suggested and I
have simplified the SendFiles function. Client can only request the regular files, any other kind such as
directories or symlinks will be skipped, the client will be responsible for taking care of such.


--
Asif Rehman
Highgo Software (Canada/China/Pakistan)
URL : www.highgo.ca

Attachment

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Wrong assert in TransactionGroupUpdateXidStatus
Next
From: Mark Dilger
Date:
Subject: Re: Make autovacuum sort tables in descending order of xid_age