Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes - Mailing list pgsql-hackers

From Dilip Kumar
Subject Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Date
Msg-id CAFiTN-v4nh2E2AA91e6aBjVV6fxWvm=33g_guExYuyqq9GSL5A@mail.gmail.com
Whole thread Raw
In response to Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>)
Responses Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>)
List pgsql-hackers
On Thu, Dec 30, 2021 at 1:41 PM Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:

>
> Yeah, that's true, but even if we are blocking the transactions from committing then also it is possible that a new connection can come and generate more WAL,  yeah but I agree with the other part that if you throttle after committing then the user can cancel the queries and generate more WAL from those sessions as well.  But that is an extreme case where application developers want to bypass the throttling and want to generate more WALs.

How about having the new hook at the start of the new txn?  If we do
this, when the limit for the throttling is exceeded, the current txn
(even if it is a long running one) continues to do the WAL insertions,
the next txns would get blocked. Thoughts?

Do you mean while StartTransactionCommand or while assigning a new transaction id?  If it is at StartTransactionCommand then we would be blocking the sessions which are only performing read queries right?  If we are doing at the transaction assignment level then we might be holding some of the locks so this might not be any better than throttling inside the commit.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Bharath Rupireddy
Date:
Subject: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Next
From: Bharath Rupireddy
Date:
Subject: Re: warn if GUC set to an invalid shared library