Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint
Date
Msg-id CABUevEzQh1Mo1rrb5pknL6=bAn_agK-1YxHHyJNPZ9FOS=iHcQ@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting forcheckpoint  (Michael Banck <michael.banck@credativ.de>)
Responses Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint
Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint
List pgsql-hackers
On Mon, Feb 13, 2017 at 10:33 AM, Michael Banck <michael.banck@credativ.de> wrote:
Hi,

Am Montag, den 13.02.2017, 09:31 +0100 schrieb Magnus Hagander:
> On Mon, Feb 13, 2017 at 3:29 AM, Jim Nasby <Jim.Nasby@bluetreble.com>
> wrote:
>         On 2/11/17 4:36 AM, Michael Banck wrote:
>                 I guess you're right, I've moved it further down.
>                 There is in fact a
>                 message about the xlog location (unless you switch off
>                 wal entirely),
>                 but having another one right before that mentioning
>                 the completed
>                 checkpoint sounds ok to me.
>
>         1) I don't think this should be verbose output. Having a
>         program sit there "doing nothing" for no apparent reason is
>         just horrible UI design.
>
>
> That would include much of Unix then.. For example if I run "cp" on a
> large file it sits around "doing nothing". Same if I do "tar". No?

The expectation for all three commands is that, even if there is no
output on stdout, they will write data to the local machine. So you can
easily monitor the progress of cp and tar by running du or something in
a different terminal.

With pg_basebackup, nothing is happening on the local machine until the
checkpoint on the remote is finished; while this is obvious to somebody
familiar with how basebackups work internally, it appears to be not
clear at all to some users.

True.

However, outputing this info by default will make it show up in things like everybodys cronjobs by default. Right now a successful pg_basebackup run will come out with no output at all, which is how most Unix commands work, and brings it's own advantages. If we change that people will have to send all the output to /dev/null, resulting in missing the things that are actually important in any regard.
 

So I think notifying the user that something is happening remotely while
the local process waits would be useful, but on the other hand,
pg_basebackup does not print anything unless (i) --verbose is set or
(ii) there is an error, so I think having it mention the checkpoint in
--verbose mode only is acceptable.

Yeah, that's my view as well. I'm all for including it in verbose mode.

*Iff* we can get a progress indicator through the checkpoint we could include that in --progress mode. But that's a different patch, of course, but it shouldn't be included in the default output even if we find it.

--

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] Parallel Append implementation
Next
From: Robert Haas
Date:
Subject: Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove all references to "xlog"from SQL-callable functions in p