Re: renice on an I/O bound box - Mailing list pgsql-performance

Willy-Bas Loos wrote:

On Tue, Jan 19, 2010 at 2:28 PM, Jean-David Beyer <jeandavid8@verizon.net <mailto:jeandavid8@verizon.net>> wrote:

   It could make sense.

   I once had a job populating a database. It was I/O bound and ran for a
   couple of hours. In those days, it was on a machine shared with
   developers of other software, and running my job interfered a lot with
   their work. I tried running it "nice" and this slowed my job down a lot
   so it took something like 16 hours. But it did not help the other users
   because my job was screwing the disk cache in memory that much longer.
   It worked out best for everyone to run my job at normal priority, but to
   start it just before quitting time so it was nearly finished by the time
   they came back to work.

   There was a better way to solve the problem (use raw IO for the dbms),
   but in those days they would not let me do that. I had done it before
   when I was working in another department and it worked just fine.

   It is my understanding that postgreSQL does not use raw IO because it
   does not help all that much. With my current machine with 8 GB RAM (6 GB
   of which is is cache) and a relatively small database, this seems to be
   true.


Did you mean to not send that to the list?

I meant to send it to the list. Most of the lists I am on do that
automatically when I press Reply. A few do not, and I forget which is which.

Seems like what you are saying is that in your case, it didn't work out.
The cache thing is a good one to remember.

How could it make sense then?

Because the process screws up the disk cache, and that takes time to
recover. At which point your process runs again and messes up the cache
again.

I have a similar problem now. I run a lot of BOINC processes at the
lowest priority (nice 19) to eat up all the processor cycles. These
normally do not interfere with anything. But when I run a postgreSQL
process, it runs slowly because whenever postgreSQL pauses for IO, the
BOINC stuff runs and invalidates (in this case, the memory L1, L2, and
L3) cache, so when postgreSQL gets a processor, all the memory caches
are invalid and the machine runs at RAM speeds (533MHz) instead of cache
speed (3.06 GHz). So if I turn off the BOINC processes, that do not
compete for processor cycles, but do compete for cache slots, the
postgreSQL process runs faster.

--
 .~.  Jean-David Beyer          Registered Linux User 85642.
 /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org

 ^^-^^ 09:00:01 up 11 days, 10:55, 3 users, load average: 4.34, 4.42, 4.43