Re: Backup throttling - Mailing list pgsql-hackers

From PostgreSQL - Hans-Jürgen Schönig
Subject Re: Backup throttling
Date
Msg-id 58C79999-3EBC-4378-88BB-A0EB5045E782@cybertec.at
Whole thread Raw
In response to Re: Backup throttling  (Andres Freund <andres@2ndquadrant.com>)
Responses Re: Backup throttling  (Andres Freund <andres@2ndquadrant.com>)
List pgsql-hackers
On Aug 19, 2013, at 9:11 PM, Andres Freund wrote:

> On 2013-08-19 20:15:51 +0200, Boszormenyi Zoltan wrote:
>> 2013-08-19 19:20 keltezéssel, Andres Freund írta:
>>> Hi,
>>>
>>> On 2013-07-24 09:20:52 +0200, Antonin Houska wrote:
>>>> Hello,
>>>> the purpose of this patch is to limit impact of pg_backup on running server.
>>>> Feedback is appreciated.
>>> Based on a quick look it seems like you're throttling on the receiving
>>> side. Is that a good idea? Especially over longer latency links, TCP
>>> buffering will reduce the effect on the sender side considerably.
>
>> Throttling on the sender side requires extending the syntax of
>> BASE_BACKUP and maybe START_REPLICATION so both can be
>> throttled but throttling is still initiated by the receiver side.
>
> Seems fine to me. Under the premise that the idea is decided to be
> worthwile to be integrated. Which I am not yet convinced of.

i think there is a lot of value for this one. the scenario we had a couple of times is pretty simple:
just assume a weak server - maybe just one disk or two - and a slave.
master and slave are connected via a 1 GB network.
pg_basebackup will fetch data full speed basically putting those lonely disks out of business.
we actually had a case where a client asked if "PostgreSQL is locked during base backup". of
course it was just disk wait caused by a full speed pg_basebackup.

regarding the client side implementation: we have chosen this way because it is less invasive.
i cannot see a reason to do this on the server side because we won't have 10
pg_basebackup-style tools making use of this feature anyway.

of course, if you got 20 disk and a 1 gbit network this is useless - but many people don't have that.
regards,
    hans-jürgen schönig


--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de




pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])
Next
From: Andres Freund
Date:
Subject: Re: Backup throttling