Thread: is the base backup protocol used by out-of-core tools?
Hi, Commit 0ba281cb4bf9f5f65529dfa4c8282abb734dd454 added a new syntax for the BASE_BACKUP command, and commit cc333f32336f5146b75190f57ef587dff225f565 added a new COPY subprotocol for taking base backups. In both cases, I preserved backward compatibility. However, nothing in PostgreSQL itself cares about this, because if you try to run an older version of pg_basebackup against a v15 server, it's going to fail anyway: pg_basebackup: error: incompatible server version 15devel From that point of view, there's no downside to removing from the server the old syntax for BASE_BACKUP and the old protocol for taking backups. We can't remove anything from pg_basebackup, because it is our practice to make new versions of pg_basebackup work with old versions of the server. But the reverse is not true: an older pg_basebackup will categorically refuse to work with a newer server version. Therefore keeping the code for this stuff around in the server has no value ... unless there is out-of-core code that (a) uses the BASE_BACKUP command and (b) wouldn't immediately adopt the new syntax and protocol anyway. If there is, we might want to keep the backward-compatibility code around in the server for a few releases. If not, we should probably nuke that code to simplify things and reduce the maintenance burden. Patches for the nuking are attached. If nobody writes back, I'm going to assume that means nobody cares, and commit these some time before feature freeze. If one or more people do write back, then my plan is to see what they have to say and proceed accordingly. Thanks, -- Robert Haas EDB: http://www.enterprisedb.com
Attachment
On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote: > Patches for the nuking are attached. If nobody writes back, I'm going > to assume that means nobody cares, and commit these some time before > feature freeze. If one or more people do write back, then my plan is > to see what they have to say and proceed accordingly. I created this and added that for visibility and so it's not forgotten. ISTM that could be done post-freeze, although I don't know if that's useful or important. https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items -- Justin
On Tue, Feb 8, 2022 at 1:03 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > I created this and added that for visibility and so it's not forgotten. > ISTM that could be done post-freeze, although I don't know if that's useful or > important. > > https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items Thanks. I feel like this is in the department of things that are unlikely to be an issue for anyone, but could be. But the set of people who could care is basically just backup tool authors, so hopefully they'll notice this thread, or that list. -- Robert Haas EDB: http://www.enterprisedb.com
On 2/8/22 12:39, Robert Haas wrote: > On Tue, Feb 8, 2022 at 1:03 PM Justin Pryzby <pryzby@telsasoft.com> wrote: >> I created this and added that for visibility and so it's not forgotten. >> ISTM that could be done post-freeze, although I don't know if that's useful or >> important. >> >> https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items > > Thanks. I feel like this is in the department of things that are > unlikely to be an issue for anyone, but could be. But the set of > people who could care is basically just backup tool authors, so > hopefully they'll notice this thread, or that list. I'm aware of several tools that use pg_basebackup but they are using the command-line tool, not the server protocol directly. Personally, I'm in favor of simplifying the code on the server side. If anyone is using the protocol directly (which I kind of doubt) I imagine they would want to take advantage of all the new features that have been added. Regards, -David
On Tue, Feb 8, 2022 at 1:52 PM David Steele <david@pgmasters.net> wrote: > I'm aware of several tools that use pg_basebackup but they are using the > command-line tool, not the server protocol directly. Good to know. > Personally, I'm in favor of simplifying the code on the server side. If > anyone is using the protocol directly (which I kind of doubt) I imagine > they would want to take advantage of all the new features that have been > added. I agree, and I'm glad you do too, but I thought it was worth asking. -- Robert Haas EDB: http://www.enterprisedb.com
On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote: > From that point of view, there's no downside to removing from the > server the old syntax for BASE_BACKUP and the old protocol for taking > backups. We can't remove anything from pg_basebackup, because it is > our practice to make new versions of pg_basebackup work with old > versions of the server. But the reverse is not true: an older > pg_basebackup will categorically refuse to work with a newer server > version. Therefore keeping the code for this stuff around in the > server has no value ... unless there is out-of-core code that (a) uses > the BASE_BACKUP command and (b) wouldn't immediately adopt the new > syntax and protocol anyway. If there is, we might want to keep the > backward-compatibility code around in the server for a few releases. > If not, we should probably nuke that code to simplify things and > reduce the maintenance burden. This line of arguments looks sensible from here, so +1 for this cleanup in the backend as of 15~. I am not sure if we should worry about out-of-core tools that use replication commands, either, and the new grammar is easy to adapt to. FWIW, one backup tool maintained by NTT is pg_rman, which does not use the replication protocol AFAIK: https://github.com/ossc-db/pg_rman Perhaps Horiguchi-san or Fujita-san have an opinion on that. -- Michael
Attachment
2022年2月9日(水) 11:21 Michael Paquier <michael@paquier.xyz>: > > On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote: > > From that point of view, there's no downside to removing from the > > server the old syntax for BASE_BACKUP and the old protocol for taking > > backups. We can't remove anything from pg_basebackup, because it is > > our practice to make new versions of pg_basebackup work with old > > versions of the server. But the reverse is not true: an older > > pg_basebackup will categorically refuse to work with a newer server > > version. Therefore keeping the code for this stuff around in the > > server has no value ... unless there is out-of-core code that (a) uses > > the BASE_BACKUP command and (b) wouldn't immediately adopt the new > > syntax and protocol anyway. If there is, we might want to keep the > > backward-compatibility code around in the server for a few releases. > > If not, we should probably nuke that code to simplify things and > > reduce the maintenance burden. > > This line of arguments looks sensible from here, so +1 for this > cleanup in the backend as of 15~. I am not sure if we should worry > about out-of-core tools that use replication commands, either, and the > new grammar is easy to adapt to. FWIW the out-of-core utility I have some involvement in and which uses base backup functionality, uses this by executing pg_basebackup, so no problem there. But even if it used the replication protocol directly, it'd just be a case of adding another bit of Pg version-specific code handling; I certainly wouldn't expect core to hold itself back for my convenience. Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
At Wed, 9 Feb 2022 11:21:41 +0900, Michael Paquier <michael@paquier.xyz> wrote in > On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote: > > From that point of view, there's no downside to removing from the > > server the old syntax for BASE_BACKUP and the old protocol for taking > > backups. We can't remove anything from pg_basebackup, because it is > > our practice to make new versions of pg_basebackup work with old > > versions of the server. But the reverse is not true: an older > > pg_basebackup will categorically refuse to work with a newer server > > version. Therefore keeping the code for this stuff around in the > > server has no value ... unless there is out-of-core code that (a) uses > > the BASE_BACKUP command and (b) wouldn't immediately adopt the new > > syntax and protocol anyway. If there is, we might want to keep the > > backward-compatibility code around in the server for a few releases. > > If not, we should probably nuke that code to simplify things and > > reduce the maintenance burden. > > This line of arguments looks sensible from here, so +1 for this > cleanup in the backend as of 15~. I am not sure if we should worry > about out-of-core tools that use replication commands, either, and the > new grammar is easy to adapt to. > > FWIW, one backup tool maintained by NTT is pg_rman, which does not use > the replication protocol AFAIK: > https://github.com/ossoc-db/pg_rman > Perhaps Horiguchi-san or Fujita-san have an opinion on that. # Oh, the excessive 'o' perplexed me:p Thanks for pining. AFAICS, as you see, pg_rman doesn't talk basebackup protocol (nor even pg_basebackup command) as it supports inremental backup. So there's no issue about the removal of old syntax on our side. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote: > Thanks for pining. AFAICS, as you see, pg_rman doesn't talk > basebackup protocol (nor even pg_basebackup command) as it supports > inremental backup. So there's no issue about the removal of old > syntax on our side. Cool. Since the verdict here seems pretty unanimous and we've heard from a good number of backup tool authors, I'll give this another day or so and then commit, unless objections emerge before then. -- Robert Haas EDB: http://www.enterprisedb.com
Am Mittwoch, dem 09.02.2022 um 08:01 -0500 schrieb Robert Haas:
> On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi
> <horikyota.ntt@gmail.com> wrote:
> > Thanks for pining. AFAICS, as you see, pg_rman doesn't talk
> > basebackup protocol (nor even pg_basebackup command) as it supports
> > inremental backup. So there's no issue about the removal of old
> > syntax on our side.
>
> Cool. Since the verdict here seems pretty unanimous and we've heard
> from a good number of backup tool authors, I'll give this another day
> or so and then commit, unless objections emerge before then.
>
A while ago i worked on a tool which talks the streaming protocol
directly. In the past there were already additions to the protocol
(like MANIFEST), so to support those optional features and multiple
major versions you already have to abstract the code to deal with that.
Having said that, i don't have any objections either to remove the old
protocol support.
Thanks
Bernd
On Wed, Feb 9, 2022 at 2:02 PM Robert Haas <robertmhaas@gmail.com> wrote: > > On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi > <horikyota.ntt@gmail.com> wrote: > > Thanks for pining. AFAICS, as you see, pg_rman doesn't talk > > basebackup protocol (nor even pg_basebackup command) as it supports > > inremental backup. So there's no issue about the removal of old > > syntax on our side. > > Cool. Since the verdict here seems pretty unanimous and we've heard > from a good number of backup tool authors, I'll give this another day > or so and then commit, unless objections emerge before then. Late to the game, but here's another +1. -- Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/
On Wed, Feb 9, 2022 at 3:05 PM Magnus Hagander <magnus@hagander.net> wrote: > Late to the game, but here's another +1. Not that late, thanks for chiming in. Looks pretty unanimous, so I have committed the patches. -- Robert Haas EDB: http://www.enterprisedb.com