Thread: scaling beyond 4 processors
Hello everyone! Since our current Postgres server, a quad Xeon system, finally can't keep up with our load anymore we're ready to take the next step. So the question is: Has anyone experiences with running Postgres on systems with more than 4 processors in a production environment? Which systems and architectures are you using (e.g. IBM xseries, IBM pseries, HP Proliant, Sun Fire, 8- way Opteron)? How about conflicts between Postgres' shared memory approach and the NUMA architecture of most multi-processor machines? Maybe it's time to switch to Oracle or DB2, but before I give up on Postgres, I wanted to hear some other opinions. Thanks for any hints and suggestions. Best regards, Stephan Vogler CipSoft GmbH
On Dec 6, 2004, at 5:18 PM, vogler@cipsoft.com wrote: > Hello everyone! > > Since our current Postgres server, a quad Xeon system, finally can't > keep up with our > load anymore we're ready to take the next step. > I'm assuming you've already done as much query tweaking as possible. and are you sure you are CPU bound and not IO bound? (Symptoms of IO bound are low cpu usage, high load average, poor performance. Many processes in "D" state) > So the question is: Has anyone experiences with running Postgres on > systems with > more than 4 processors in a production environment? Which systems and Have you also considered a replicated approach? -- Jeff Trout <jeff@jefftrout.com> http://www.jefftrout.com/ http://www.stuarthamm.net/
In the last exciting episode, vogler@cipsoft.com wrote:
> Hello everyone!
>
> Since our current Postgres server, a quad Xeon system, finally can't
> keep up with our load anymore we're ready to take the next step.
>
> So the question is: Has anyone experiences with running Postgres on
> systems with more than 4 processors in a production environment?
> Which systems and architectures are you using (e.g. IBM xseries, IBM
> pseries, HP Proliant, Sun Fire, 8- way Opteron)? How about conflicts
> between Postgres' shared memory approach and the NUMA architecture
> of most multi-processor machines?
The perhaps odd thing is that just about any alternative to quad-Xeon
is likely to be _way_ better. There are some context switching
problems that lead to it being remarkably poorer than you'd expect.
Throw in less-than ideal performance of the PAE memory addressing
system and it seems oddly crippled overall.
We've been getting pretty good results with IBM pSeries systems;
they're expensive, but definitely very fast.
Preliminary results with Opterons are also looking very promising.
One process seemed about 25x as fast on a 4-way 8GB Opteron as it was
on a 4-way 8GB Xeon, albeit with enough differences to make the
comparison dangerous.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://www.ntlug.org/~cbbrowne/sgml.html
The IETF motto: "Rough consensus *and* working code."
The perhaps odd thing is that just about any alternative to quad-Xeon
is likely to be _way_ better. There are some context switching
problems that lead to it being remarkably poorer than you'd expect.
Throw in less-than ideal performance of the PAE memory addressing
system and it seems oddly crippled overall.
We've been getting pretty good results with IBM pSeries systems;
they're expensive, but definitely very fast.
Preliminary results with Opterons are also looking very promising.
One process seemed about 25x as fast on a 4-way 8GB Opteron as it was
on a 4-way 8GB Xeon, albeit with enough differences to make the
comparison dangerous.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://www.ntlug.org/~cbbrowne/sgml.html
The IETF motto: "Rough consensus *and* working code."
Volger, > Since our current Postgres server, a quad Xeon system, finally can't keep > up with our load anymore we're ready to take the next step. There are a lot of reasons this could be happening; Quad Xeon is a problematic platform, the more so if you're on Dell hardware. I've run PostgreSQL on 8-ways, and I know there are a few Sunfire users around the community (16-way). There are definitely specific performance issues around specific query loads on multi-way systems; search this list archives for "Context Switch Bug". I will echo others in saying that moving to Opteron on premium hardware should jump you at least 2x on performance, and it's a lot cheaper than DB2 or Oracle. And, of course, if you really want help from this list, you'll post more specific problems. -- Josh Berkus Aglio Database Solutions San Francisco