Re: pgbench unable to scale beyond 100 concurrent connections - Mailing list pgsql-hackers
| From | Sachin Kotwal |
|---|---|
| Subject | Re: pgbench unable to scale beyond 100 concurrent connections |
| Date | |
| Msg-id | CA+N_YAewy+cWWZiiC2KhQo5Hgz870Xw8RQBr-cyjBdQVMLgcYA@mail.gmail.com Whole thread Raw |
| In response to | Re: pgbench unable to scale beyond 100 concurrent connections (Fabien COELHO <coelho@cri.ensmp.fr>) |
| Responses |
Re: pgbench unable to scale beyond 100 concurrent connections
(Robert Haas <robertmhaas@gmail.com>)
Re: pgbench unable to scale beyond 100 concurrent connections (Jeff Janes <jeff.janes@gmail.com>) Re: pgbench unable to scale beyond 100 concurrent connections (Fabien COELHO <coelho@cri.ensmp.fr>) |
| List | pgsql-hackers |
Hi Fabien,
--
Thanks and Regards,
Sorry for very short report.
I feel pgbench is not so complex tool.
Please see below answers to your questions.
On Wed, Jun 29, 2016 at 5:07 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
Hello Sachin,
Your report is very imprecise so it is hard to tell anything.
What version of client and server are you running?
I am testing it with 9.6-beta1 binaries. For server and client it is same.
I am using pgbench on top of postgres_fdw.
On what hardware ? (200 connections => 200 active postgres processes, how many processes per core are you expecting to run?
I am running in small virtual box machine. with 1GB RAM and 2 cores.
I think there should not be problem with 200 processes on 2 core machines.
I tested same number of concurrent connections on same machine with sysbench it is working fine.
I am not sure what is difference between pgbench and sysbench, they might be process based or thread based.
But I can say if I can create 200+ concurrent connection with sysbench , i should be able create same with pgbench.
Thoughts?
the recommanded value is about 2 connections per physical core...)
I think 2 connections per core is very small value . for 200 i need atleast 100 core machine , which is not good.
What precise command is started?
How to you know it "comes down to 100 connections"?
I put watch on live connections to database.
something like : watch -n 1 'ps -ef | grep postgres | grep 192.168.56.101 | wc -l'
NOTE: grep cxommand may change as per environment.
Also user below query to see active connection.
# select count(*) from pg_stat_activity;
Are there error messages from pgbench or postgresql?
postgresql does not give any error.
pgbench says:
client 36 aborted in state 2: ERROR: could not connect to server "server_1"
DETAIL: FATAL: sorry, too many clients already
My random guess would be that you start too many connections with only one thread client side and/or on a too small hardware client or server-side for the expected scale, so given the load and latency some connections just never get to do anything?
This may be reason but it should be able to maintain idle connection for that time if never get to do anything.
Maybe try with "-j 20" so that there are not too many connections per pgbench thread?
I do not have such good hardware for now.
I feel pgbench should be able to perform well on small hardware.
Feel free to ask any question regarding setup.
I am testing pgbench with more than 100 connections. also set max_connection in postgresql.conf more than 100.
Initially pgbench tries to scale nearby 150 but later it come down to 100
connections and stable there.
It this limitation of pgbench? or bug? or i am doing it wrong way?
---
I tested it with max_connection = 200 in postgresql.conf
and pgbench witn -c 180/190/200Please reply.
Please send precise information instead of expecting people to guess...
--
Fabien
Sachin Kotwal
pgsql-hackers by date: