Re: Database performs massive reads when I'm doing writes. - Mailing list pgsql-general

From: "Ray Cote" <rgacote@appropriatesolutions.com>
To: "Jeff Janes" <jeff.janes@gmail.com>
Cc: pgsql-general@postgresql.org
Sent: Thursday, June 6, 2013 5:12:51 PM
Subject: Re: [GENERAL] Database performs massive reads when I'm doing writes.

p { margin: 0; }

It is Postmaster itself:
11068                 -              315.9M               136K                  0K               85%             postmaster
11000                 -              56808K                 8K                  0K               15%             postmaster 
11003                 -                  0K                80K                  0K                0%             postmaster
11004                 -                  0K                24K                  0K                0%             postmaster
11067                 -                  0K                 0K                  0K                0%             load_rets.py

The above are the atop lines for just postmaster. This is a 10s snapshot so you can see lots of read activity.
The load_rets.py task is the Python script loading the database.

--Ray

---

From: "Jeff Janes" <jeff.janes@gmail.com>
To: "Ray Cote" <rgacote@appropriatesolutions.com>
Cc: pgsql-general@postgresql.org
Sent: Thursday, June 6, 2013 4:05:28 PM
Subject: Re: [GENERAL] Database performs massive reads when I'm doing writes.

On Thu, Jun 6, 2013 at 9:53 AM, Ray Cote <rgacote@appropriatesolutions.com> wrote:

Hello:

I have a PostgreSQL 9.0.3 database that has suddenly started exhibiting odd read behavior.

The version number is:
"PostgreSQL 9.0.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50), 64-bit"
This was compiled from source about a year ago and has been running just fine.
The database feeds a very low-volume web site (a few hits per minute).

Starting a few days ago we started to see a strange behavior where writing to the database causes massive read operations.
For example, I have a table that needs to be updated every night (about 20,000 rows).
Using Django ORM, we update them one item at a time.
Usually the overall process takes a few minutes; it is now taking hours (like over 15 hours).

Running atop, we're seeing Read Disk values in the range of 147.2M/10s and Write Disk values in the range of 16K/10s.
Together, the Disk throughput is in the high 90% and frequently hits 100%.

Which specific processes are using the disk?  (on my atop, if I hit 'c' it will change mode to show me the entire so-called "command line", which will let you know if the culprit is a vacuum worker, the checkpointer, or someone else).

Cheers,

Jeff

--

Ray Cote, President Appropriate Solutions, Inc.
We Build Software
www.AppropriateSolutions.com 603.924.6079