What would people here think about changing the default value of
client_connection_check_interval to 2000 ms? Right now this is disabled
by default.
The background is that I recently saw an incident where a blocking-lock
brownout escalated from a row-level problem to a complete system
outage, due to a combination of factors including a bug in golang's pgx
postgres client (PR 2481 has now been merged w a fix) and a pgbouncer
setup that was missing peers configuration. As a result, cancel
messages were getting dropped while postgres connections were waiting
on a blocked lock, golang aggresively timed out on context deadlines
and retried, and once the database reached max_connections the whole
system ground to a halt.
At the time I thought it was weird that postgres wasn't checking for
dead connections while those conns were waiting for locks; I spent a
bunch of time investigating this and reproduced it and wrote up what I
was able to figure out.
Then, yesterday, I saw a LinkedIn post from Marat at Data Egret who
mentioned that client_connection_check_interval exists.
Plugged this into my repro and confirmed it can prevent postgres from
escalating the blocking-lock brownout into a complete outage due to
connection exhaustion.
While a fix has been merged in pgx for the most direct root cause of
the incident I saw, this setting just seems like a good behavior to
make Postgres more robust in general. 2000 ms seemed like a fairly
safe/conservative starting point for discussion. Thoughts?
-Jeremy
PS. Some more details and graphs are at
https://ardentperf.com/2026/02/04/postgres-client_connection_check_interval/
--
To know the thoughts and deeds that have marked man's progress is to
feel the great heart throbs of humanity through the centuries; and if
one does not feel in these pulsations a heavenward striving, one must
indeed be deaf to the harmonies of life.
Helen Keller, The Story Of My Life, 1902, 1903, 1905, introduction by
Ralph Barton Perry (Garden City, NY: Doubleday & Company, 1954), p90.
