Thread: Re: Commit Latency
Hello On 2025-Feb-05, Ramakrishna m wrote: > I have a system handling *300 TPS*, with resource usage *below 10%*. > However, I’m noticing *commit latency of around 200ms* for *1% of > transactions*, occasionally spiking to *1 second*. Since there is no > significant *I/O pressure*, I’m trying to identify what else might be > causing this in *PostgreSQL 16*. max_connections=8000 doesn't sound great -- how many of those are active, typically, and how many are idle-in-transaction? And you have autovacuum_naptime=5s ... which sounds rather dubious. Either somebody with great expertise configured this very carefully, or the settings are somewhat randomly chosen with little or no expert oversight. Do you have monitoring on the amount of bloat on these database? Maybe you should consider connection pooling and limit the number that are active, for starters. Maybe have a look at whether pg_wait_sampling can give you more clues. Some basic bloat monitoring is a prerequisite to any further performance tweaking anyhow. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ Maybe there's lots of data loss but the records of data loss are also lost. (Lincoln Yeoh)
Thanks for the suggestions!
It looks the issue is happening at the time of wal creation, does wal_init_zero off is good option?
Best
On Wed, 5 Feb, 2025, 9:07 pm Álvaro Herrera, <alvherre@alvh.no-ip.org> wrote:
Hello
On 2025-Feb-05, Ramakrishna m wrote:
> I have a system handling *300 TPS*, with resource usage *below 10%*.
> However, I’m noticing *commit latency of around 200ms* for *1% of
> transactions*, occasionally spiking to *1 second*. Since there is no
> significant *I/O pressure*, I’m trying to identify what else might be
> causing this in *PostgreSQL 16*.
max_connections=8000 doesn't sound great -- how many of those are
active, typically, and how many are idle-in-transaction? And you have
autovacuum_naptime=5s ... which sounds rather dubious. Either somebody
with great expertise configured this very carefully, or the settings are
somewhat randomly chosen with little or no expert oversight. Do you
have monitoring on the amount of bloat on these database? Maybe you
should consider connection pooling and limit the number that are active,
for starters.
Maybe have a look at whether pg_wait_sampling can give you more clues.
Some basic bloat monitoring is a prerequisite to any further performance
tweaking anyhow.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Maybe there's lots of data loss but the records of data loss are also lost.
(Lincoln Yeoh)
On 2/7/25 14:00, ravi k wrote: > Thanks for the suggestions! > > It looks the issue is happening at the time of wal creation, does > wal_init_zero off is good option? > It's impossible to say if that option will help, considering how little info about the system you provided. I suggest you start by answering the questions you've been asked by me & Alvaro, explain why you concluded it's related to WAL creation, etc. regards -- Tomas Vondra