scaling multiple connections - Mailing list pgsql-hackers

From mlw
Subject scaling multiple connections
Date
Msg-id 3AE816FD.68BE88EB@mohawksoft.com
Whole thread Raw
Responses Re: scaling multiple connections  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: scaling multiple connections  (Lincoln Yeoh <lyeoh@pop.jaring.my>)
List pgsql-hackers
I am getting a bit concerned about Postgres 7.1 performance with multiple
connections. Postgres does not seem to scaling very well. Below there is a list
of outputs from pgbench with different number of clients, you will see that
postgres' performance in the benchmark drops with each new connection.
Shouldn't the tps stay fairly constant?

I am using pgbench because I saw this performance issue with a project I was
developing. I decided to try doing operations in parallel, thinking that
postgres would scale. What I found was the more machines I added to the task,
the slower the processing was. 

Anyone have any ideas? Is this how it is supposed to be?

My postmaster start line looks like:
/usr/local/pgsql/bin/postmaster -A0 -N 24 -B 4096 -i -S -D/sqlvol/pgdev -o -F
-fs -S 2048

The database is on a dedicated PCI-IDE/66 promise card, with a 5400rpm maxtor
drive, not the best hardware, I grant you, but that should have little to do
with the scaling aspect.

I am running redhat linux 7.0, kernel 2.4.3. 512M ram, dual PIII 600mhz.

[markw@snoopy pgbench]$ ./pgbench -v  -c 1  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 1
number of transactions per client: 30
number of transactions actually processed: 30/30
tps = 218.165952(including connections establishing)
tps = 245.062001(excluding connections establishing)
[markw@snoopy pgbench]$ ./pgbench -v  -c 2  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 2
number of transactions per client: 30
number of transactions actually processed: 60/60
tps = 200.861024(including connections establishing)
tps = 221.175326(excluding connections establishing)
[markw@snoopy pgbench]$ ./pgbench -v  -c 3  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 3
number of transactions per client: 30
number of transactions actually processed: 90/90
tps = 144.053242(including connections establishing)
tps = 154.083205(excluding connections establishing)
[markw@snoopy pgbench]$ ./pgbench -v  -c 4  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 4
number of transactions per client: 30
number of transactions actually processed: 120/120
tps = 129.709537(including connections establishing)
tps = 137.852284(excluding connections establishing)
[markw@snoopy pgbench]$ ./pgbench -v  -c 5  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 5
number of transactions per client: 30
number of transactions actually processed: 150/150
tps = 103.569559(including connections establishing)
tps = 108.535287(excluding connections establishing)


.......

[markw@snoopy pgbench]$ ./pgbench -v  -c 20  -t 30 pgbench
starting vacuum...end.
starting full vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 20
number of transactions per client: 30
number of transactions actually processed: 600/600
tps = 40.600209(including connections establishing)
tps = 41.352773(excluding connections establishing)


pgsql-hackers by date:

Previous
From: Mauricio Breternitz
Date:
Subject: concurrent postgres in NUMA cluster postgres - design OK ?
Next
From: "Len Morgan"
Date:
Subject: Re: refusing connections based on load ...