Re: WAL Rate Limiting - Mailing list pgsql-hackers

From Pavan Deolasee
Subject Re: WAL Rate Limiting
Date
Msg-id CABOikdNi7eZm-iB5a-7GOcQP4L1Mm3tXJTRcy7UMKc9QNJzs_w@mail.gmail.com
Whole thread Raw
In response to WAL Rate Limiting  (Simon Riggs <simon@2ndQuadrant.com>)
List pgsql-hackers



On Wed, Jan 15, 2014 at 7:50 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
We've discussed previously the negative impact of large bulk
operations, especially wrt WAL writes. Patch here allows maintenance
operations to have their WAL generation slowed down as a replication
lag prevention feature.

I believe there was originally intended to be some work on I/O rate
limiting, but that hasn't happened and is in some ways orthogonal to
this patch and we will likely eventually want both.

Single new parameter works very similarly to vacuum_cost_delay

wal_rate_limit_delay = Xms

slows down CLUSTER, VACUUM FULL, ALTER TABLE (rewrite & set
tablespace), CREATE INDEX
so basically same things we optimise WAL for and the same places where
we honour maintenance_work_mem
(discuss: should we add COPY, CTAS etc also?)
(discuss: do we need another parameter to specify "cost"? Currently
patch uses "sleep every 64kB of WAL")

VACUUM is not included, since we already have controls for that -
honouring two controls would be complex and weird.


I wonder if should replace vacuum_cost/delay with say maintenance_cost/delay and use it in all maintenance activities including what you just listed out above. While the exact semantics of vacuum and other maintenance activities may differ, ISTM the final goal is just the same i.e. reduce load on the master.

Thanks,

pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: Add CREATE support to event triggers
Next
From: Peter Eisentraut
Date:
Subject: tests for client programs