Thread: scaling up postgres

Hello,

I am setting up a postgres server that will hold a critical event witin
the next few weeks.
It's national exam result (140000 students)
the problem is that the first few hours there will be a huge traffic,
(last year 250K requests only the first hour)
I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
One beast will be apache, and the other will be postgres.
I'm using httperf/autobench for measurments and the best result I can get
is that my system can handle a trafiic of almost 1600 New con/sec.

I cannot scale beyond that value and the funny thing, is that none of the
servers is swapping, or heavy loaded, neither postgres nor apache are
refusing connexions.
my database is only 58M it's a read only DB and will lasts only for a month.
here is my pstgres.conf :
---------------------------- postgres.conf - begin
max_connections = 6000
shared_buffers = 12288
work_mem = 512
maintenance_work_mem = 16384
effective_cache_size = 360448
random_page_cost     2
log_destination     'stderr'
redirect_stderr    on
log_min_messages    notice
log_error_verbosity    default
log_disconnections     on
autovacuum    off
stats_start_collector    off
stats_row_level    off
---------------------------- postgres.conf - end
---------------------------- sysctl.conf (postgres) - begin
fs.file-max=    5049800
net.ipv4.ip_local_port_range=   1024 65000
net.ipv4.tcp_keepalive_time=    120
kernel.shmmax=  2147483648
kernel.sem= 250 96000 100 384
---------------------------- sysctl.conf (postgres) - end
kernel semaphores are grown as postgres needs.

vmstat is not showing any "annoyance" I cannot find where is it blocked !

please, I need help as it's a very critical deployment :(

NB : apache when stressed for a static page, i can handle more 16k new con/sec

On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
> I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
> One beast will be apache, and the other will be postgres.
> I'm using httperf/autobench for measurments and the best result I can get
> is that my system can handle a trafiic of almost 1600 New con/sec.

What version of PostgreSQL? (8.1 is better than 8.0 is much better than 7.4.)
Have you remembered to turn HT off? Have you considered Opterons instead of
Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind of queries
are you running? Are you using connection pooling? Prepared queries?

> vmstat is not showing any "annoyance" I cannot find where is it blocked !

How many context switches ("cs" in vmstat) do you get per second?

/* Steinar */
--
Homepage: http://www.sesse.net/

> One beast will be apache, and the other will be postgres.

> I'm using httperf/autobench for measurments and the best result I can get
> is that my system can handle a trafiic of almost 1600 New con/sec.

> NB : apache when stressed for a static page, i can handle more 16k new
> con/sec

    That's not the point.
    Here are a few points of advice.

    USE LIGHTTPD DAMMIT !

    If you want performance, that is.
    On my server (Crap Celeron) it handles about 100 hits/s on static files
and 10 hits/s on PHP pages ; CPU utilization is 1-2%, not counting PHP.
    lighttpd handles 14K static pages/s on my laptop. That's about as much as
your bi-xeon does with apache...

    You want a web server that uses as little CPU as possible so that more
CPU is left for generating dynamic content.

    Also, you want to have a number of concurrent DB connections which is
neither too high, nor too low.
    Apache + mod_php needs to spawn a lot of processes, thus you need a lot
of database connections.
    This tends not to be optimal.

    Too few concurrent DB connections -> network latency between DB and
webserver will be a bottleneck.
    Too many connections -> excess context switching, suboptimal use of CPU
cache, memory use bloat.

    So, I advise you to use lighttpd fronting PHP as fastcgi (if you use PHP)
; if you use Java or whatever which has a HTTP interface, use lighttpd as
a proxy for your dynamic page generation.
    Spawn a reasonable number of PHP processes. The number depends on your
application, but from 10 to 30 is a good starting point.

    USE PERSISTENT DATABASE CONNECTIONS !

    Postgres will breathe a little better ; now, check if it is still slow.
If it is, you need to find the bottleneck...
    I can help you a bit by private email if you want.

>I cannot scale beyond that value and the funny thing, is that none of the
>servers is swapping, or heavy loaded, neither postgres nor apache are
>refusing connexions.
>
>
Hearing a story like this (throughput hits a hard limit, but
hardware doesn't appear to be 100% utilized), I'd suspect
insufficient concurrency to account for the network latency
between the two servers.

Also check that your disks aren't saturating somewhere (with
iostat or something similar).

You could run pstack against both processes and see what they're
doing while the system is under load. That might give a clue
(e.g. you might see the apache processs waiting on
a response from PG, and the PG processes waiting
on a new query to process).

Since you've proved that your test client and apache can
handle a much higher throughput, the problem must lie
somewhere else (in posgresql or the interface between
the web server and postgresql).

fzied@planet.tn writes:
> I'm using httperf/autobench for measurments and the best result I can get
> is that my system can handle a trafiic of almost 1600 New con/sec.

As per PFC's comment, if connections/sec is a bottleneck for you then
the answer is to use persistent connections.  Launching a new backend
is a fairly heavyweight operation in Postgres.  It sounds like there
may be some system-level constraints affecting the process creation
rate as well, but it's silly to spend a lot of effort on this when
you can so easily go around the problem.

            regards, tom lane

fzied@planet.tn writes: 

I'm using httperf/autobench for measurments and the best result I can get 
is that my system can handle a trafiic of almost 1600 New con/sec.   

As per PFC's comment, if connections/sec is a bottleneck for you then
the answer is to use persistent connections.  Launching a new backend
is a fairly heavyweight operation in Postgres.  

>
> Tom Lane wrote:
> As per PFC's comment, if connections/sec is a bottleneck for you then
the
> answer is to use persistent connections. Launching a new backend
is a fairly
> heavyweight operation in Postgres.
>

In which case maybe pgpool could help in this respect?
http://pgpool.projects.postgresql.org/

Cheers,

Neil.

On Sun, Jun 11, 2006 at 11:42:20PM +0200, Mario Splivalo wrote:
> Could you point out to some more detailed reading on why Xeons are
> poorer choice than Opterons when used with PostgreSQL?

There are lots of theories, none conclusive, but the benchmarks certainly
point that way consistently. Read the list archives for the details.

/* Steinar */
--
Homepage: http://www.sesse.net/

On Sat, 2006-06-03 at 11:43 +0200, Steinar H. Gunderson wrote:
> On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
> > I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
> > One beast will be apache, and the other will be postgres.
> > I'm using httperf/autobench for measurments and the best result I can get
> > is that my system can handle a trafiic of almost 1600 New con/sec.
>
> What version of PostgreSQL? (8.1 is better than 8.0 is much better than 7.4.)
> Have you remembered to turn HT off? Have you considered Opterons instead of
> Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind of queries

Could you point out to some more detailed reading on why Xeons are
poorer choice than Opterons when used with PostgreSQL?

    Mario

Mario Splivalo wrote:
> On Sat, 2006-06-03 at 11:43 +0200, Steinar H. Gunderson wrote:
>> On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
>>> I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
>>> One beast will be apache, and the other will be postgres.
>>> I'm using httperf/autobench for measurments and the best result I can get
>>> is that my system can handle a trafiic of almost 1600 New con/sec.
>> What version of PostgreSQL? (8.1 is better than 8.0 is much better than 7.4.)
>> Have you remembered to turn HT off? Have you considered Opterons instead of
>> Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind of queries
>
> Could you point out to some more detailed reading on why Xeons are
> poorer choice than Opterons when used with PostgreSQL?

It isn't just PostgreSQL. It is any database. Opterons can move memory
and whole lot faster then Xeons.

Joshua D. Drake

>
>     Mario
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>


--

             === The PostgreSQL Company: Command Prompt, Inc. ===
       Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
       Providing the most comprehensive  PostgreSQL solutions since 1997
                      http://www.commandprompt.com/

Hi Mario,

I did run pgbench on several production servers:
HP DL585 - 4-way AMD Opteron 875
HP DL585 - 4-way AMD Opteron 880
HP DL580 G3 - 4-way Intel XEON MP 3.0 GHz
FSC RX600 S2 - 4-way Intel XEON MP DC 2.66 GHz
FSC RX600 - 4-way Intel XEON MP 2.5 GHz

This test has been done with 8.1.4. I increased the number of clients.
I attached the result as diagram. I included not all test system but the
gap between XEON and Opteron is always the same.

The experiences with production systems were the same. We replaced the
XEON box with Opteron box with a dramatic change of performance.

Best regards
Sven.


Mario Splivalo schrieb:
> On Sat, 2006-06-03 at 11:43 +0200, Steinar H. Gunderson wrote:
>> On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
>>> I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
>>> One beast will be apache, and the other will be postgres.
>>> I'm using httperf/autobench for measurments and the best result I can get
>>> is that my system can handle a trafiic of almost 1600 New con/sec.
>> What version of PostgreSQL? (8.1 is better than 8.0 is much better than 7.4.)
>> Have you remembered to turn HT off? Have you considered Opterons instead of
>> Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind of queries
>
> Could you point out to some more detailed reading on why Xeons are
> poorer choice than Opterons when used with PostgreSQL?
>
>     Mario
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster

--
/This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you are not the intended recipient, you should not
copy it, re-transmit it, use it or disclose its contents, but should
return it to the sender immediately and delete your copy from your
system. Thank you for your cooperation./

Sven Geisler <sgeisler@aeccom.com> Tel +49.30.5362.1627 Fax .1638
Senior Developer,    AEC/communications GmbH    Berlin,   Germany

On 12 Jun 2006, at 00:21, Joshua D. Drake wrote:

> Mario Splivalo wrote:
>> On Sat, 2006-06-03 at 11:43 +0200, Steinar H. Gunderson wrote:
>>> On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
>>>> I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
>>>> One beast will be apache, and the other will be postgres.
>>>> I'm using httperf/autobench for measurments and the best result
>>>> I can get is that my system can handle a trafiic of almost 1600
>>>> New con/sec.
>>> What version of PostgreSQL? (8.1 is better than 8.0 is much
>>> better than 7.4.)
>>> Have you remembered to turn HT off? Have you considered Opterons
>>> instead of
>>> Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind
>>> of queries
>> Could you point out to some more detailed reading on why Xeons are
>> poorer choice than Opterons when used with PostgreSQL?
>
> It isn't just PostgreSQL. It is any database. Opterons can move
> memory and whole lot faster then Xeons.

A whole lot faster indeed.

http://www.amd.com/us-en/Processors/ProductInformation/
0,,30_118_8796_8799,00.html
http://www.theinquirer.net/?article=10797

Although apparently the dual core ones are a little better than the
old ones

http://www.anandtech.com/IT/showdoc.aspx?i=2644

(Just to provide some evidence ;)

Hi all,

Joshua D. Drake schrieb:
> Mario Splivalo wrote:
>> Could you point out to some more detailed reading on why Xeons are
>> poorer choice than Opterons when used with PostgreSQL?
>
> It isn't just PostgreSQL. It is any database. Opterons can move memory
> and whole lot faster then Xeons.

Yes. You can run good old memtest86 and you see the difference.
Here my numbers with memtest86 (blocksize 128 MB).
HP DL580 G3 (4-way XEON MP - DDR RAM) => 670 MByte/sec
FSC RX600 S2 (4-way XEOM MP DC - DDR2-400 PC2-3200) => 1300 MByte/sec
HP DL585 (4-way Opteron DDR2-400 PC2-3200) => 1500 MByte/sec
I used memxfer5b.cpp.

Cheers Sven.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sven Geisler wrote:
> Hi Mario,
>
> I did run pgbench on several production servers:
> HP DL585 - 4-way AMD Opteron 875
> HP DL585 - 4-way AMD Opteron 880
> HP DL580 G3 - 4-way Intel XEON MP 3.0 GHz
> FSC RX600 S2 - 4-way Intel XEON MP DC 2.66 GHz
> FSC RX600 - 4-way Intel XEON MP 2.5 GHz
>
> This test has been done with 8.1.4. I increased the number of clients.
> I attached the result as diagram. I included not all test system but the
> gap between XEON and Opteron is always the same.
>
> The experiences with production systems were the same. We replaced the
> XEON box with Opteron box with a dramatic change of performance.
>
> Best regards
> Sven.
>
>
> Mario Splivalo schrieb:
>> On Sat, 2006-06-03 at 11:43 +0200, Steinar H. Gunderson wrote:
>>> On Sat, Jun 03, 2006 at 10:31:03AM +0100, fzied@planet.tn wrote:
>>>> I do have 2 identical beasts (4G - biproc Xeon 3.2 - 2 Gig NIC)
>>>> One beast will be apache, and the other will be postgres.
>>>> I'm using httperf/autobench for measurments and the best result I
>>>> can get is that my system can handle a trafiic of almost 1600 New
>>>> con/sec.
>>> What version of PostgreSQL? (8.1 is better than 8.0 is much better
>>> than 7.4.)
>>> Have you remembered to turn HT off? Have you considered Opterons
>>> instead of
>>> Xeons? (The Xeons generally scale bad with PostgreSQL.) What kind of
>>> queries
>>
>> Could you point out to some more detailed reading on why Xeons are
>> poorer choice than Opterons when used with PostgreSQL?
>>
>>     Mario
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 2: Don't 'kill -9' the postmaster
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
Thank you for sharing this.
Coming back to my problem :) A very faithful partner accepted to
gracefully borrow us 3 Pseries (bi-ppc + 2G RAM not more). with linux on
them.
Now I'm trying to make my tests, and I'm not that sure I will make the
switch to the PSeries, since my dual xeon with 4 G RAM can handle 3500
concurrent postmasters consuming 3.7 G of the RAM. I cannot reach this
number on the PSeries with 2 G.

can someone give me advice ?
BTW, I promise, at the end of my tests, I'll publish my report.

- --
Zied Fakhfakh
GPG Key : gpg --keyserver subkeys.pgp.net  --recv-keys F06B55B5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFEjeDbS1DO7ovpKz8RAnLGAJ96/1ndGoc+HhBvOfrmlQnJcfxa6QCfQK9w
i6/GGUCBGk5pdNUDAmVN5RQ=
=5Mns
-----END PGP SIGNATURE-----

On Mon, Jun 12, 2006 at 11:47:07PM +0200, Zydoon wrote:
> Thank you for sharing this.
> Coming back to my problem :) A very faithful partner accepted to
> gracefully borrow us 3 Pseries (bi-ppc + 2G RAM not more). with linux on
> them.
> Now I'm trying to make my tests, and I'm not that sure I will make the
> switch to the PSeries, since my dual xeon with 4 G RAM can handle 3500
> concurrent postmasters consuming 3.7 G of the RAM. I cannot reach this
> number on the PSeries with 2 G.
>
> can someone give me advice ?

Uhm... stick with commodity CPUs?

Seriously, unless you're going to run on some seriously big hardware
you'll be hard-pressed to find better performance/dollar than going with
a server running Opterons.

If you're trying to decide how much hardware you need to meet a specific
performance target there's a company here in Austin I can put you in
touch with; if I'm not mistaken on the cost of pSeries hardware their
fee would be well worth it to make sure you don't end up with too much
(or worse, too little) hardware for your load.

> BTW, I promise, at the end of my tests, I'll publish my report.

Great. More performance data is always good to have.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

On Mon, 2006-06-12 at 16:47, Zydoon wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Thank you for sharing this.
> Coming back to my problem :) A very faithful partner accepted to
> gracefully borrow us 3 Pseries (bi-ppc + 2G RAM not more). with linux on
> them.
> Now I'm trying to make my tests, and I'm not that sure I will make the
> switch to the PSeries, since my dual xeon with 4 G RAM can handle 3500
> concurrent postmasters consuming 3.7 G of the RAM. I cannot reach this
> number on the PSeries with 2 G.
>
> can someone give me advice ?
> BTW, I promise, at the end of my tests, I'll publish my report.

Search the performance archives for the last 4 or 5 months for PPC /
pseries machines.

You'll find a very long thread about the disappointing performance the
tester got with a rather expensive P Series machine.  And his happy
ending of testing on an Opteron machine.

On Tue, 13 Jun 2006 14:28:49 -0500
Scott Marlowe <smarlowe@g2switchworks.com> wrote:

> Search the performance archives for the last 4 or 5 months for PPC /
> pseries machines.
>
> You'll find a very long thread about the disappointing performance the
> tester got with a rather expensive P Series machine.  And his happy
> ending of testing on an Opteron machine.

Amen, brother :)

We hoped throwing a silly pSeries 650 would solve all our problems. Boy
were we wrong... a world of pain...

Don't go there - just buy an Opteron system - if you're talking about
IBM big iron, a decent Opteron will cost you about as much as a couple
of compilers and an on-site visit from IBM...

Cheers,
Gavin.

> Uhm... stick with commodity CPUs?

    Hehe, does this include Opterons ?

    Still, I looked on the "customize your server" link someone posted and
it's amazing ; these things have become cheaper while I wasn't looking...
    You can buy 10 of these boxes with raptors and 4 opteron cores and 8 gigs
of RAM for the price of your average marketing boss's car... definitely
makes you think doesn't it.

    Juts wait until someone equates the price in man-hours to fix/run a
borken Dell box...

On Tue, Jun 13, 2006 at 10:14:44PM +0200, PFC wrote:
>
> >Uhm... stick with commodity CPUs?
>
>     Hehe, does this include Opterons ?

Absolutely. Heck, it wouldn't surprise me if a single model # of Opteron
sold more than all Power CPUs put together...

>     Still, I looked on the "customize your server" link someone posted
>     and  it's amazing ; these things have become cheaper while I wasn't
> looking...
>     You can buy 10 of these boxes with raptors and 4 opteron cores and 8
>     gigs  of RAM for the price of your average marketing boss's car...
> definitely  makes you think doesn't it.

And if you spend that much on CPU for a database, you're likely to be
pretty sadly disappointed, depending on what you're doing.

>     Juts wait until someone equates the price in man-hours to fix/run a
> borken Dell box...

Would probably sound like a Mastercard commercial...

Not having to babysit your servers every day: Priceless
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gavin Hamill wrote:
> On Tue, 13 Jun 2006 14:28:49 -0500
> Scott Marlowe <smarlowe@g2switchworks.com> wrote:
>
>> Search the performance archives for the last 4 or 5 months for PPC /
>> pseries machines.
>>
>> You'll find a very long thread about the disappointing performance the
>> tester got with a rather expensive P Series machine.  And his happy
>> ending of testing on an Opteron machine.
>
> Amen, brother :)
>
> We hoped throwing a silly pSeries 650 would solve all our problems. Boy
> were we wrong... a world of pain...
>
> Don't go there - just buy an Opteron system - if you're talking about
> IBM big iron, a decent Opteron will cost you about as much as a couple
> of compilers and an on-site visit from IBM...
>
> Cheers,
> Gavin.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq
>
Things cannot be clearer :)
I really know that opterons are the best I can have.
But for now I have to publish the results Sunday 25th on either the
xeons or the PPCs.
Tomorrow I'll conduct a deeper test, and come back.
Cheers.

- --
Zied Fakhfakh
GPG Key : gpg --keyserver subkeys.pgp.net  --recv-keys F06B55B5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFEjy0cS1DO7ovpKz8RAklqAKDC75a8SQUoGwNHGxu4ysZhNt5eJwCgt0mP
YHfZbYVS44kxFyxxEzs9KE0=
=aLbn
-----END PGP SIGNATURE-----

On Tue, Jun 13, 2006 at 05:40:58PM -0400, John Vincent wrote:
> Maybe from a postgresql perspective the cpus may be useless but the memory
> on the pSeries can't be beat. We've been looking at running our warehouse
> (PGSQL) in a LoP lpar but I wasn't able to find a LoP build of 8.1.

Probably just because not many people have access to that kind of
hardware. Have you tried building on Linux on Power?

Also, I believe Opterons can do up to 4 DIMMs per memory controller, so
with 2G sticks an 8 way Opteron could hit 64GB, which isn't exactly
shabby, and I suspect it'd cost quite a bit less than a comperable
p570...

> We've been thrilled with the performance of our DB2 systems that run on
> AIX/Power 5 but since the DB2 instance memory is limited to 18GB, we've got
> two 86GB p570s sitting there being under utilized.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

On Tue, Jun 13, 2006 at 06:21:21PM -0400, John Vincent wrote:
> On 6/13/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
> >
> >On Tue, Jun 13, 2006 at 05:40:58PM -0400, John Vincent wrote:
> >> Maybe from a postgresql perspective the cpus may be useless but the
> >memory
> >> on the pSeries can't be beat. We've been looking at running our
> >warehouse
> >> (PGSQL) in a LoP lpar but I wasn't able to find a LoP build of 8.1.
> >
> >Probably just because not many people have access to that kind of
> >hardware. Have you tried building on Linux on Power?
>
>
> Actually it's on my radar. I was looking for a precompiled build first (we
> actually checked the Pervasive and Bizgres sites first since we're
> considering a support contract) before going the self-compiled route. When I
> didn't see a pre-compiled build available, I started looking at the
> developer archives and got a little worried that I wouldn't want to base my
> job on a self-built Postgres on a fairly new (I'd consider Power 5 fairly
> new) platform.

Well, pre-compiled isn't going to make much of a difference
stability-wise. What you will run into is that very few people are
running PostgreSQL on your hardware, so it's possible you'd run into
some odd corner cases. I think it's pretty unlikely you'd lose data, but
you could end up with performance-related issues.

If you can, it'd be great to do some testing on that hardware to see if
you can break PostgreSQL.

> This is true. In our case I couldn't get the approval for the new hardware
> since we had two x445 boxes sitting there doing nothing (I wanted them for
> our VMware environment personally). Another sticking point is finding a
> vendor that will provide a hardware support contract similar to what we have
> with our existing IBM hardware (24x7x4). Since IBM has f-all for Opteron
> based systems and we've sworn off Dell, I was pretty limited. HP was able to
> get in on a pilot program and we're considering them now for future hardware
> purchases but beyond Dell/IBM/HP, there's not much else that can provide the
> kind of hardware support turn-around we need.

What about Sun?

> >We've been thrilled with the performance of our DB2 systems that run on
> >> AIX/Power 5 but since the DB2 instance memory is limited to 18GB, we've
> >got
> >> two 86GB p570s sitting there being under utilized.

BTW, in a past life we moved a DB2 database off of Xeons and onto
RS/6000s with Power4. The difference was astounding.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

"Jim C. Nasby" <jnasby@pervasive.com> writes:
> On Tue, Jun 13, 2006 at 06:21:21PM -0400, John Vincent wrote:
>> Actually it's on my radar. I was looking for a precompiled build first (we
>> actually checked the Pervasive and Bizgres sites first since we're
>> considering a support contract) before going the self-compiled route. When I
>> didn't see a pre-compiled build available, I started looking at the
>> developer archives and got a little worried that I wouldn't want to base my
>> job on a self-built Postgres on a fairly new (I'd consider Power 5 fairly
>> new) platform.

> Well, pre-compiled isn't going to make much of a difference
> stability-wise. What you will run into is that very few people are
> running PostgreSQL on your hardware, so it's possible you'd run into
> some odd corner cases.

Power 5 is just a PPC64 platform isn't it?  Red Hat's been building PG
for PPC64 for years, and I've not heard any problems reported.  Now,
if you're using a non-gcc compiler then maybe that track record doesn't
carry over to whatever you are using ...

            regards, tom lane

Hi, Fzied,

fzied@planet.tn wrote:

> I'm using httperf/autobench for measurments and the best result I can
> get is that my system can handle a trafiic of almost 1600 New
> con/sec.

Are you using connection pooling or persistent connections between
PostgreSQL and the Apaches?

Maybe it simply is the network latency between the two machines - as the
database is read-only, did you think about having both PostgreSQL and
Apache on both machines, and then load-balancing ingoing http requests
between them?

> I cannot scale beyond that value and the funny thing, is that none of
> the servers is swapping, or heavy loaded, neither postgres nor apache
> are refusing connexions.

And for measuring, are you really throwing parallel http connections to
the server? This sounds like you measure request latency, but the
maximum throughput might be much higher.

> my database is only 58M it's a read only DB and will lasts only for a
> month.

I guess it is a simple table with a single PK (some subscription numer)
- no joins or other things.

For this cases, a special non-RDBMS like MySQL, SQLite, or even some
hancrafted thingy may give you better results.


Markus


--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf.     | Software Development GIS

Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org

Hi, Zydoon,

Zydoon wrote:

> Now I'm trying to make my tests, and I'm not that sure I will make the
> switch to the PSeries, since my dual xeon with 4 G RAM can handle 3500
> concurrent postmasters consuming 3.7 G of the RAM. I cannot reach this
> number on the PSeries with 2 G.

This sounds like you want to have one postgresql backend per apache
frontend.

Did you try running pgpool on the Apache machine, and have only a few
(hundred) connections to the backend?

Maybe http://en.wikipedia.org/wiki/Memcached could be helpful, too.


Markus

--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf.     | Software Development GIS

Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org