Re: Backup throttling - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Backup throttling
Date
Msg-id 521B4B29.20009@2ndquadrant.com
Whole thread Raw
In response to Re: Backup throttling  (Hannu Krosing <hannu@2ndQuadrant.com>)
Responses Re: Backup throttling  (Antonin Houska <antonin.houska@gmail.com>)
List pgsql-hackers
On 08/26/2013 08:15 PM, Hannu Krosing wrote:
> On 08/26/2013 12:50 PM, Antonin Houska wrote:
>> > On 08/22/2013 03:33 PM, Craig Ringer wrote:
>>> >> On 08/22/2013 01:39 PM, PostgreSQL - Hans-Jürgen Schönig wrote:
>>> >>
>>>> >>> what would be a reasonable scenario where limiting streaming would make sense? i cannot think of any to be
honest.
>>> >> I tend to agree. If anything we're likely to want the reverse - the
>>> >> ability to throttle WAL *generation* on the master so streaming can keep up.
>> > (I assume you mean WAL *sending* rather than *generation*.)
> I think he meant *generation*, that is making sure that no more
> than X amount of WAL is generated in Y amount of time, thereby
> making sure that you can stream less WAL without falling behind.
>

Exactly so.

Sometimes one doesn't want the latency of synchronous replication, but
still wants to set a limit on how far behind the standby can fall. It
might be for limiting data loss, for making sure a replica can keep up
sustainably, or just to make sure the replica never gets far enough
behind that the master discards WAL that it still requires.

For many people the idea of the replica(s) affecting the master is
horrifying. For others, missing a single transaction on a crash is
unthinkable. We handle both those pretty well, but the middle ground is
currently very hard to walk, and I think that's actually where many
people will want to be.

I'd certainly rather throttle the master (and send a loud, angry alert
to the admin via icinga/nagios) than have a replica fall too far behind
and need to be re-inited from scratch.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services



pgsql-hackers by date:

Previous
From: Oleg Bartunov
Date:
Subject: Re: Hstore: Query speedups with Gin index
Next
From: Pavel Stehule
Date:
Subject: median and percentile function implementation