Re: [BUGS] BUG #14831: Intermittent write blocks - Mailing list pgsql-bugs

From Andres Freund
Subject Re: [BUGS] BUG #14831: Intermittent write blocks
Date
Msg-id 20170927210610.7w6dn7swxiltp3ci@alap3.anarazel.de
Whole thread Raw
In response to [BUGS] BUG #14831: Intermittent write blocks  (harisankar.valiyattil@capitalone.com)
Responses Re: [BUGS] BUG #14831: Intermittent write blocks
List pgsql-bugs
Hi,

On 2017-09-27 20:59:33 +0000, harisankar.valiyattil@capitalone.com wrote:
> The following bug has been logged on the website:
> 
> Bug reference:      14831
> Logged by:          Harisankar Valiyattil
> Email address:      harisankar.valiyattil@capitalone.com
> PostgreSQL version: 9.4.7
> Operating system:   linux
> Description:        
> 
> We use postgres on amazon RDS. We have a table with close to 15 million data
> now and an application is constantly firing some update(insert/update)
> operations to this table at around 25 transactions per second. the table has
> a jsonb column which has a gin index on it; the jonb data has appx 20-40
> attrinbutes in an average. The application keep on updating data on this
> jsonb column.
> 
> While the traffic is running, we see that there is a sudden drop in the
> write operations (no write happening in DB) causing increase in RDS cpu,
> queue depth etc. after a few seconds (30 sec, 60 sec) things come back to
> normal. If the traffic is constant this repeats at constant intervals.
> 
> Our work_mem is 4MB. 
> 
> What could be causing this? What kind of troubleshooting we can do? What is
> the process that runs in the background that causes this intermittant write
> delay?

Presumably it's the OS starting to flush dirty buffers to disk, which it
periodically does. Not sure if RDS allows you to monitor that, but the
Dirt and Writeback lines in /proc/meminfo would show that.  9.6+ have
code to limit the amount of dirty buffers in the kernel (enabled by
default for background processes, disabled by default for backends
themseleves).

Greetings,

Andres Freund


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

pgsql-bugs by date:

Previous
From: harisankar.valiyattil@capitalone.com
Date:
Subject: [BUGS] BUG #14831: Intermittent write blocks
Next
From: "Valiyattil, Harisankar"
Date:
Subject: Re: [BUGS] BUG #14831: Intermittent write blocks