On Fri, Jan 10, 2014 at 3:56 AM, Mika Eloranta <mel@ohmu.fi> wrote:
On Thu, Jan 9, 2014, at 20:05, Magnus Hagander wrote:
> On Thu, Nov 14, 2013 at 10:27 AM, Mika Eloranta <mel@ohmu.fi> wrote: >> On 13 Nov 2013, at 20:51, Mika Eloranta <mel@ohmu.fi> wrote: >> >> >> > Prevent excessive progress reporting that can grow to gigabytes >> > of output with large databases. >> >> >> Same patch as an attachment. > > Would it not make more sense to instead store the last number printed, and only print it if the percentage has changed? AIUI with this patch we still print the same thing on top of itself if it takes >1 second to get 1% further. > > (Except for verbose mode - but if you're asking for verbose mode, you are *asking* to get lots of output)
(re-sent response as I used the wrong sender address previously, sorry about the dupe)
Printing out progress periodically is IMHO slightly better as the interactive user can see that there is some progress (e.g. by pressing enter for a new empty console line) during a huge basebackup operation.
That's an argument I hadn't considered - but I still think it's acceptable to wait until the next percentage digit in this case.
The original problem I wanted to address was that I had a daemon watching over the basebackup process and reading its output to make sure that the basebackup is proceeding properly. It also wrote all the output to a logfile for postmortem analysis. The log file grew to a very big size (multiple gigabytes due to the progress prints). With my patch the log was only a few kilos, without any negative effects. The excessive progress reporting can also be an issue in an interactive session over a slow network (mobile), hogging both time and bandwidth.
Yeah, I was guessing it was something like that. I didn't realize you'd actually monitor it through a deamon (I've just looked at the output filesize when minitoring things like that), but the remote connection was easily reproducible. I definitely agree we should do something about it.
