Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Date
Msg-id 20211229193151.tq7b34cdmnds6w6t@alap3.anarazel.de
Whole thread Raw
In response to Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>)
Responses Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>)
Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Hi,

On 2021-12-27 16:40:28 -0800, SATYANARAYANA NARLAPURAM wrote:
> > Yet another problem is that if we are in XlogInsert() that means we are
> > holding the buffer locks on all the pages we have modified, so if we add a
> > hook at that level which can make it wait then we would also block any of
> > the read operations needed to read from those buffers.  I haven't thought
> > what could be better way to do this but this is certainly not good.
> >
> 
> Yes, this is a problem. The other approach is adding a hook at
> XLogWrite/XLogFlush?

That's pretty much the same - XLogInsert() can trigger an
XLogWrite()/Flush().

I think it's a complete no-go to add throttling to these places. It's quite
possible that it'd cause new deadlocks, and it's almost guaranteed to have
unintended consequences (e.g. replication falling back further because
XLogFlush() is being throttled).

I also don't think it's a sane thing to add hooks to these places. It's
complicated enough as-is, adding the chance for random other things to happen
during such crucial operations will make it even harder to maintain.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: SATYANARAYANA NARLAPURAM
Date:
Subject: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Next
From: SATYANARAYANA NARLAPURAM
Date:
Subject: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes