Thread: Instability with incremental backup tests (pg_combinebackup, 003_timeline.pl)

Hi all,

dikkop has reported a failure with the regression tests of pg_combinebackup:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dikkop&dt=2024-08-04%2010%3A04%3A51

That's in the test 003_timeline.pl, from dc212340058b:
#   Failed test 'incremental backup from node1'
#   at t/003_timeline.pl line 43.

The node is extremely slow, so perhaps bumping up the timeout would be
fine enough in this case (did not spend time analyzing it).  I don't
think that this has been discussed, but perhaps I just missed a
reference to it and the incremental backup thread is quite large.

Thanks,
--
Michael

On 8/6/24 07:48, Michael Paquier wrote:
> Hi all,
> 
> dikkop has reported a failure with the regression tests of pg_combinebackup:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dikkop&dt=2024-08-04%2010%3A04%3A51
> 
> That's in the test 003_timeline.pl, from dc212340058b:
> #   Failed test 'incremental backup from node1'
> #   at t/003_timeline.pl line 43.
> 
> The node is extremely slow, so perhaps bumping up the timeout would be
> fine enough in this case (did not spend time analyzing it).  I don't
> think that this has been discussed, but perhaps I just missed a
> reference to it and the incremental backup thread is quite large.
> 

Yeah, it's a freebsd running on rpi4, from a USB flash disk, and in my
experience it's much slower than rpi4 running Linux. I'm not sure why is
that, never found a way to make it faster

The machine already has:

  export PGCTLTIMEOUT=600
  export PG_TEST_TIMEOUT_DEFAULT=600

I doubt increasing it further will do the trick. Maybe there's some
other timeout that I should increase?

FWIW I just moved the buildfarm stuff to a proper SSD disk (still USB,
but hopefully better than the crappy flash disk).


regards

-- 
Tomas Vondra

On 8/6/24 14:53, Tomas Vondra wrote:
> On 8/6/24 07:48, Michael Paquier wrote:
>> Hi all,
>>
>> dikkop has reported a failure with the regression tests of pg_combinebackup:
>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dikkop&dt=2024-08-04%2010%3A04%3A51
>>
>> That's in the test 003_timeline.pl, from dc212340058b:
>> #   Failed test 'incremental backup from node1'
>> #   at t/003_timeline.pl line 43.
>>
>> The node is extremely slow, so perhaps bumping up the timeout would be
>> fine enough in this case (did not spend time analyzing it).  I don't
>> think that this has been discussed, but perhaps I just missed a
>> reference to it and the incremental backup thread is quite large.
>>
> 
> Yeah, it's a freebsd running on rpi4, from a USB flash disk, and in my
> experience it's much slower than rpi4 running Linux. I'm not sure why is
> that, never found a way to make it faster
> 
> The machine already has:
> 
>   export PGCTLTIMEOUT=600
>   export PG_TEST_TIMEOUT_DEFAULT=600
> 
> I doubt increasing it further will do the trick. Maybe there's some
> other timeout that I should increase?
> 
> FWIW I just moved the buildfarm stuff to a proper SSD disk (still USB,
> but hopefully better than the crappy flash disk).
> 

Seems the move to SSD helped a lot - the runs went from ~4h to ~40m. So
chances are the instability won't be such a problem.

regards

-- 
Tomas Vondra