[HACKERS] Slow I/O can break throttling of base backup - Mailing list pgsql-hackers

From Magnus Hagander
Subject [HACKERS] Slow I/O can break throttling of base backup
Date
Msg-id CABUevEy_-e0YvL4ayoX8bH_Ja9w+BHoP6jUgdxZuG2nEj3uAfQ@mail.gmail.com
Whole thread Raw
Responses Re: [HACKERS] Slow I/O can break throttling of base backup  (Antonin Houska <ah@cybertec.at>)
List pgsql-hackers
Running pg_basebackup with a throttling of say 10M runs it into the risk of the I/O on the server actually being slower than pg_basebackup (I have preproduced similar issues on fake-slow disks with lower rate limits).

What happens in this case in basebackup.c is that the value for "sleep" comes out negative. This means we don't sleep, which is fine.

However, that also means we don't set throttle_last.

That means that the next time we come around to throttle(), the value for "elapsed" is even bigger, which results in an even bigger negative number, and we're "stuck".

AFAICT this means that a temporary I/O spike that makes reading of the disk too slow can leave us in a situation where we never recover, and thus never end up sleeping ever again, effectively turning off rate limiting.

I wonder if the 
else if (sleep > 0)
at the bottom of throttle()
should just be a simple else and always run, resetting last_throttle?


--

pgsql-hackers by date:

Previous
From: David Fetter
Date:
Subject: Re: [HACKERS] Transaction oddity with list partition of a listpartition
Next
From: Magnus Hagander
Date:
Subject: [HACKERS] Missing newlines in error messages