Unfortunately, the buildfarm log doesn't contain regress_log_002_limits, but I managed to reproduce the failure on that my device, when it's storage as slow as: $ dd if=/dev/zero of=./test count=1024 oflag=dsync bs=128k 1024+0 records in 1024+0 records out 134217728 bytes (134 MB, 128 MiB) copied, 33.9446 s, 4.0 MB/s
The past ~1 week, I tried to space out all other tasks on the machine, so as to ensure
that 1-min CPU is mostly <2 (and thus not many things hammering the disk) and with
that I see 0 failures these past few days. This isn't conclusive by any means, but it
does seem that reducing IO contention has helped remove the errors, like what
Alexander suspects / repros here.
Just a note, that I've reverted some of those recent changes now, and so if the theory
holds true, I wouldn't be surprised if some of these errors restarted on dodo.
