Re: high io BUT huge amount of free memory - Mailing list pgsql-hackers

From Andres Freund
Subject Re: high io BUT huge amount of free memory
Date
Msg-id 20130424141811.GF4536@awork2.anarazel.de
Whole thread Raw
In response to Re: high io BUT huge amount of free memory  (Shaun Thomas <sthomas@optionshouse.com>)
Responses Re[2]: [HACKERS] high io BUT huge amount of free memory
List pgsql-hackers
On 2013-04-24 09:06:39 -0500, Shaun Thomas wrote:
> On 04/24/2013 08:49 AM, Andres Freund wrote:
> 
> >Uh. Ranting can be rather healthy thing every now and then and it good
> >for the soul and such. But. Did you actually try reporting those issues?
> 
> That's actually part of the problem. How do you report:
> 
> Throwing a lot of processes at a high-memory system seems to break the mm
> code in horrible ways.

Well. Report memory distribution. Report perf profiles. Ask *them* what
information they need. They aren't grumpy if you are behaving
sensibly. YMMV of course.

> I'm asking seriously here, because I have no clue how to isolate this
> behavior. It's clearly happening often enough that random people are
> starting to notice now that bigger servers are becoming the norm.
> 
> I'm also not a kernel dev in any sense of the word. My C is so rusty, I can
> barely even read the patches going through the ML. I feel comfortable
> posting to PG lists because that's my bread and butter. Kernel lists seem
> way more imposing, and I'm probably not the only one who feels that way.

I can understand that. But you had to jump over the fence to post here
once as well ;). Really, report it and see what comes out. The worst
that can happen is that you get a grumpy email ;)
And in the end, jumping might ease the pain in the long run considerably
even if its uncomfortable at first...

Feel free to CC me.

> I guess I don't mean to imply that kernel devs don't care. Maybe the right
> way to put it is that there don't seem to be enough kernel devs being
> provided with more capable testing hardware. Which is odd, considering Red
> Hat's involvement and activity on the kernel.

There are quite some people using huge servers, but that doesn't imply
they are seing the same problems. During testing they mostly use a set
of a few benchmarks (part of which is pgbench btw) and apparently they
don't show this problem. Also this is horribly workload and hardware
dependent. There are enough people happily using postgres on linux on
far bigger hardware than what you reported upthread.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Shaun Thomas
Date:
Subject: Re: high io BUT huge amount of free memory
Next
From: Timothy Garnett
Date:
Subject: Allowing parallel pg_restore from pipe