Magnus Hagander <magnus@hagander.net> writes:
> Well, that certainly goes to enough detail to agree that no, that
> can't be done with only minor modifications to pg_basebackup. Nor
> could it be done with your python program talking directly to the
> walsender backend and get around it that way. But you probably already
> considered that :D
It sounds like a nice project to extend the pg_basebackup client side
prototype I did after Heikki's talk at pgcon 2010, though:
https://github.com/dimitri/pg_basebackup
This “hack” allows to take a base backup from a libpq connection without
any special protocol, so work from 8.4 onward, and by replacing a simple
enough WITH RECURSIVE form could be made to support earlier version
also.
It's already doing things in parallel with a first pass on the server to
get the list of files, you would have to enhance how the splitting is
decided then add your other features.
Well, maybe that's already what you did, without using that code… or
maybe you want to use the rsync abilities rather than libpq. Well…
A limitation of this prototype is its failure to ensure that all the WAL
produced in the meantime are actually archived in parallel of running
the base backup, which is only important if you don't have already some
archiving in place. I guess that the dedicated subprocess doing the WAL
copy could be enhanced to talk the wal streaming protocol when the
server is recent enough.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
