Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes - Mailing list pgsql-hackers

From SATYANARAYANA NARLAPURAM
Subject Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Date
Msg-id CAHg+QDcTo-p_pTPUaDNi03-be4oxDyYO5Ehai2Ec-uMcBS7J6g@mail.gmail.com
Whole thread Raw
In response to Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (Dilip Kumar <dilipbalaut@gmail.com>)
Responses Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
List pgsql-hackers


On Sat, Dec 25, 2021 at 6:01 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Dec 26, 2021 at 3:52 AM SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:


On Fri, Dec 24, 2021 at 3:13 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:


XLogInsert in my opinion is the best place to call it and the hook can be something like this "void xlog_insert_hook(NULL)" as all the throttling logic required is the current flush position which can be obtained from GetFlushRecPtr and the ReplicationSlotCtl. Attached a draft patch.

IMHO, it is not a good idea to call an external hook function inside a critical section.  Generally, we ensure that we do not call any code path within a critical section which can throw an error and if we start calling the external hook then we lose that control. 

Thank you for the comment. XLogInsertRecord is inside a critical section but not XLogInsert. Am I missing something?

Actually all the WAL insertions are done under a critical section (except few exceptions), that means if you see all the references of XLogInsert(), it is always called under the critical section and that is my main worry about hooking at XLogInsert level.

Got it, understood the concern. But can we document the limitations of the hook and let the hook take care of it? I don't expect an error to be thrown here since we are not planning to allocate memory or make file system calls but instead look at the shared memory state and add delays when required.

 
 
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Dilip Kumar
Date:
Subject: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Next
From: Julien Rouhaud
Date:
Subject: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes