Re: pg_stat_progress_basebackup - progress reporting forpg_basebackup, in the server side - Mailing list pgsql-hackers

From Fujii Masao
Subject Re: pg_stat_progress_basebackup - progress reporting forpg_basebackup, in the server side
Date
Msg-id 5191ae78-4a53-8864-3f50-4f1977b741cd@oss.nttdata.com
Whole thread Raw
In response to Re: pg_stat_progress_basebackup - progress reporting forpg_basebackup, in the server side  (Magnus Hagander <magnus@hagander.net>)
Responses Re: pg_stat_progress_basebackup - progress reporting forpg_basebackup, in the server side  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers

On 2020/03/05 9:31, Magnus Hagander wrote:
> On Mon, Mar 2, 2020 at 10:03 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>
>>
>>
>> On 2020/03/03 14:37, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote:
>>> Hi,
>>>
>>> Thank you for developing good features.
>>> The attached patch is a small fix to the committed documentation. This patch fixes the description literal for the
backup_streamedcolumn.
 
>>
>> Thanks for the report and patch! Pushed.
> 
> This patch requires, AIUI, that you add -P to the pg_basebackup
> commandline in order to get the progress tracked in details
> serverside.

Whether --progress is enabled or not, the pg_stat_progress_basebackup
view report the progress of the backup in the server side. But the total
amount of data that will be streamed is estimated and reported only when
this option is enabled.

> But this also generates output in the client that one
> might not want.
> 
> Should we perhaps have a switch in pg_basebackup that enables the
> server side tracking only, without generating output in the client?

Yes, this sounds reasonable.

I have two ideas.

(1) Extend --progress option so that it accepts the setting values like
     none, server, both (better names?). If both is specified, PROGRESS
     option is specified in BASE_BACKUP replication command and
     the total backup size is estimated in the server side, but the progress
     is not reported in a client side. If none, PROGRESS option is not
     specified in BASE_BACKUP. The demerit of this idea is that --progress
     option without argument is not supported yet and the existing
     application using --progress option when using pg_basebackup needs
     to be updated when upgrading PostgreSQL version to v13.

(2) Add something like --estimate-backup-size (better name?) option
     to pg_basebackup. If it's specified, PROGRESS option is specified but
     the progress is not reported in a client side.

Thought?

Or, as another approach, it might be worth considering to make
the server always estimate the total backup size whether --progress is
specified or not, as Amit argued upthread. If the time required to
estimate the backup size is negligible compared to total backup time,
IMO this approach seems better. If we adopt this, we can also get
rid of PROGESS option from BASE_BACKUP replication command.

Regards,


-- 
Fujii Masao
NTT DATA CORPORATION
Advanced Platform Technology Group
Research and Development Headquarters



pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Re: Identifying user-created objects
Next
From: Joe Nelson
Date:
Subject: Re: Change atoi to strtol in same place