high io BUT huge amount of free memory - Mailing list pgsql-hackers

From Миша Тюрин
Subject high io BUT huge amount of free memory
Date
Msg-id 1366654964.540342580@f106.mail.ru
Whole thread Raw
Responses Re: high io BUT huge amount of free memory
Re: high io BUT huge amount of free memory
Re: [HACKERS] high io BUT huge amount of free memory
List pgsql-hackers
My first message has been banned for too many latters.

>
Hi all
There is something wrong and ugly.

1)
Intel 32 core = 2*8 *2threads

Linux avi-sql09 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64 GNU/Linux

PostgreSQL 9.2.2 on x86_64-unknown-linux-gnu, compiled by gcc-4.4.real (Debian 4.4.5-8) 4.4.5, 64-bit
shared_buffers 64GB / constant hit rate - 99,18
max_connections 160 / with pgbouncer pools there could not be more than 120 connections at all
work_mem 32M
checkpoint 1h 1.0
swap off
numa off,  interleaving on

24*128GB HDD (RAID10) with 2GB bbu (1,5w+0,5r)

2)
free -g
             total       used       free     shared    buffers     cached
Mem:           378        250        128          0          0        229
-/+ buffers/cache:         20        357

and
! disks usage 100%  (free 128GB! WHY?)

disk throughput - up-to 30MB/s (24r+6w)
io - up-to 2,5-3K/s (0,5w + 2-2,5r)

3) so maybe I've got something like this
http://www.databasesoup.com/2012/04/red-hat-kernel-cache-clearing-issue.html
or this
http://comments.gmane.org/gmane.comp.db.sqlite.general/79457

4) now i think
a) upgrade linux core or
b) set buffers to something like 300-320Gb
my warm work set is about 300-400GB
db at all - 700GB

typical work load - pk-index-scans

--
looking forward
thanks
>
Mikhail

Attachment

pgsql-hackers by date:

Previous
From: Anne Rosset
Date:
Subject: Re: Performance with the new security release?
Next
From: David Gudeman
Date:
Subject: Re: minimizing the target list for foreign data wrappers