Re: Remove Deprecated Exclusive Backup Mode - Mailing list pgsql-hackers

On 7/1/20 3:58 PM, Robert Haas wrote:
> On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander <magnus@hagander.net> wrote:
>> As far as I've seen, the one thing that people have problems with in the exclusive mode backups are precisely the
factthat they have to keep a persistent conneciton open, and thus it cannot work together with backup software that is
limitedto only supporting running a pre- and a post script.
 
>>
>> Something like I have suggested here is to solve *that* problem. I don't think anybody actually explicitly wants
"exclusivebackups" -- they want a backup solution that plugs into their world of pre/post scripts. And if we can make
thatone work in a safer way than the current exclusive backups, ohw is that not an improvement?
 
> 
> Yeah, I guess that's a pretty fair point. I have to confess to having
> somewhat limited enthusiasm for adding a third mode here, but it might
> be worth it.
> 
> It seems pretty well inevitable to me that people are going to forget
> to end them. I am not sure exactly what the consequences of that will
> be, but if for example there's a limited number of shared memory slots
> to store information about these backups, then if you leak any, you'll
> eventually run out of slots and your backups will start failing. I
> feel like that's a going to happen to about 75% of the people who try
> to use this new backup mode at some point in time, but maybe I'm a
> pessimist.[1]

I did consider going down this road and I came to more or less the same 
conclusion. Backups would accumulate until we just ran out of space 
and/or started throwing errors at some limit. Hardly very attractive.

Also, full pages writes will be left on indefinitely.

> If we could jigger things so that you don't need to stop the backup at
> all, you only start it, and whether you ever finish copying everything
> is something about which the system need not know or care, that would
> be a lot nicer. I'm not sure I see how to do that, though.

Well, the only thing pg_stop_backup() *really* needs to know is the 
starting WAL position. pg_start_backup() gets that info so if it passes 
it back to pg_stop_backup() that could be enough. Or as was proposed 
above, it just passes the backup_label back to pg_stop_backup() for 
parsing. To write the .backup file to WAL (which I personally think is 
pretty useless) you'd need backup_label.

The issue I have not been able to work around is that full page writes 
need to be on for backups and they won't get turned back off if you 
don't end all the backups that get started.

Here's a thought. What if we just stored the oldest starting LSN and a 
count of how many backups have been requested. When the backup ends it 
checks that backup count is > 0 and starting LSN is <= its starting LSN. 
If not, it throws an error. When backups go to 0 FPWs are turned off if 
they were off before the first backup.

That way, we could have a single function to cancel all backups in 
progress. Even if a new one started, one ending that was started prior 
to the cancel would know that it could not end successfully and error.

For the vast majority of people who have full page writes on, it 
wouldn't really matter how many backups were running so we could 
probably just skip all of that.

If we wanted to really make it more foolproof we could add a checksum to 
the backup label to make sure the user doesn't damage it before passing 
it back.

Regards,
-- 
-David
david@pgmasters.net