Help troubleshooting SubtransControlLock problems - Mailing list pgsql-admin

From Scott Frazer
Subject Help troubleshooting SubtransControlLock problems
Date
Msg-id CA+ey=amOGwaBzY5UHZ50Uvo85c-tofY6j1TApuXZHHd+3RQHsA@mail.gmail.com
Whole thread Raw
Responses Re: Help troubleshooting SubtransControlLock problems  (Scott Frazer <sfrazer@couponcabin.com>)
List pgsql-admin

Hi, we have a Postgres 9.6 setup using replication that has recently started seeing a lot of processes stuck in "SubtransControlLock" as a wait_event on the read-replicas. Like this, only usually about 300-800 of them:


 179706 | LWLockNamed     | SubtransControlLock

 186602 | LWLockNamed     | SubtransControlLock

 186606 | LWLockNamed     | SubtransControlLock

 180947 | LWLockNamed     | SubtransControlLock

 186621 | LWLockNamed     | SubtransControlLock
The server then begins to crawl, with some queries just never finishing until I finally shut the server down.


Searching for that particular combo of wait_event_type and wait_event only seems to turn up the page about statistics collection, but no helpful information on troubleshooting this lock.

Restarting the replica server clears the locks and allows us to start working again, but it's happened twice now in 12 hours and I'm worried it will happen again.

Does anyone have any advice on where to start looking?

Thanks,
Scott

pgsql-admin by date:

Previous
From: Natalie Wenz
Date:
Subject: Re: pg_upgrade and frozen xids
Next
From: Mark Kirkwood
Date:
Subject: Re: Reliable WAL file shipping over unreliable network