Postgres Pro
Downloads
Home   >   mailing lists

Thread: Connection attempt hangs when server is unresponsive

Connection attempt hangs when server is unresponsive

From
"Woody Woodring"
Date:
We are using 1.2.1 of DBCP with our tomcat server to pool connections to
databases on the wide area.  We had an issue today were our tomcat threads
started increasing dramatically when we had a Postgres server go away and we
found a lot of threads stuck trying to create a datasource:

         at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.jav
a:753)
         at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:5
40)
         at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:122)
         at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:134)

Since createDataSource is synchronized we traced them to the following
thread that had them blocked:

         at java.net.PlainSocketImpl.socketConnect(Native Method)
         at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
         at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
         at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
         at java.net.Socket.connect(Socket.java:516)
         at java.net.Socket.connect(Socket.java:466)
         at java.net.Socket.(Socket.java:366)
         at java.net.Socket.(Socket.java:179)
         at org.postgresql.core.PGStream.(PGStream.java:58)
         at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFa
ctoryImpl.java:77)
         at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:
65)
         at
org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:1
17)
         at
org.postgresql.jdbc3.AbstractJdbc3Connection.(AbstractJdbc3Connection.java:3
0)
         at org.postgresql.jdbc3.Jdbc3Connection.(Jdbc3Connection.java:24)
         at org.postgresql.Driver.connect(Driver.java:235)
         at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConne
ctionFactory.java:37)
         at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnect
ionFactory.java:290)
         at
org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataS
ource.java:877)
         at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.jav
a:851)
         at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:5
40)
         at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:122)
         at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:134)

My question is why this thread did not error out when it could not talk to
the server?  Could the server have been up when the connection started and
went away in the middle causing errors?  Could there be something we have
overlooked in our conf that would time it out?  When the db server came
back, our thread count dropped from 200 down to 35 which is normal.

Thanks in advance,
Woody

----------------------------------------
iGLASS Networks
211-A S. Salem St
Apex NC 27502
(919) 387-3550 x813
www.iglass.net

Re: Connection attempt hangs when server is unresponsive

From
Oliver Jowett
Date:
Woody Woodring wrote:

> Since createDataSource is synchronized we traced them to the following
> thread that had them blocked:
>
>          at java.net.PlainSocketImpl.socketConnect(Native Method)

That's an OS-level method, if it blocks and you have not set
loginTimeout there is not much the driver can do.

Maybe you should set loginTimeout? (It is more expensive as it spawns
extra threads to deal with timeouts in this sort of case, though)

-O

Re: Connection attempt hangs when server is unresponsive

From
"Usama Dar"
Date:

My question is why this thread did not error out when it could not talk to
the server?  Could the server have been up when the connection started and
went away in the middle causing errors?  Could there be something we have
overlooked in our conf that would time it out?  When the db server came
back, our thread count dropped from 200 down to 35 which is normal.


Look into

1- Setting the jdbc driver's login timeout
2- make sure that the connection attempts are not trying to do long name resolution lookups
3- If you are using DBCP make sure tomcat is configured to remove abandoned connections (removeAbandoned=true) and related parmeters are set to reasonable value.

--
Usama Munir Dar http://linkedin.com/in/usamadar
Consultant Architect
Cell:+92 321 5020666
Skype: usamadar