Re: [HACKERS] Slow I/O can break throttling of base backup - Mailing list pgsql-hackers

From Antonin Houska
Subject Re: [HACKERS] Slow I/O can break throttling of base backup
Date
Msg-id 10526.1481818893@localhost
Whole thread Raw
In response to [HACKERS] Slow I/O can break throttling of base backup  (Magnus Hagander <magnus@hagander.net>)
Responses Re: [HACKERS] Slow I/O can break throttling of base backup
List pgsql-hackers
It seems to be my bug. I'll check tomorrow.

Magnus Hagander <magnus@hagander.net> wrote:

> Running pg_basebackup with a throttling of say 10M runs it into the risk of
> the I/O on the server actually being slower than pg_basebackup (I have
> preproduced similar issues on fake-slow disks with lower rate limits).
>
> What happens in this case in basebackup.c is that the value for "sleep"
> comes out negative. This means we don't sleep, which is fine.
>
> However, that also means we don't set throttle_last.
>
> That means that the next time we come around to throttle(), the value for
> "elapsed" is even bigger, which results in an even bigger negative number,
> and we're "stuck".
>
> AFAICT this means that a temporary I/O spike that makes reading of the disk
> too slow can leave us in a situation where we never recover, and thus never
> end up sleeping ever again, effectively turning off rate limiting.
>
> I wonder if the else if (sleep > 0) at the bottom of throttle() should just
> be a simple else and always run, resetting last_throttle?
>
> --
> Magnus Hagander
> Me: http://www.hagander.net/
> Work: http://www.redpill-linpro.com/
>

--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at



pgsql-hackers by date:

Previous
From: Ian Jackson
Date:
Subject: Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraintviolation [and 2 more messages] [and 1 more messages]
Next
From: Amit Kapila
Date:
Subject: Re: [HACKERS] Hash Indexes