Thread: org.postgresql.util.PSQLException: An I/O error has occured while flushing the output

Hi,
 I have a web application running on Tomcat4.1, using postgresql 7.5 database. Hibernate 2.1.x is being used along with jdbc driver pg74.1jdbc3.jar for mapping database.
 
 The application dumps the following error in the following cases:
   1. The postgres process is restarted.
   2. The network connection between the machine running postgres and tomcat has been reset.
 
Hibernate data source property ( hibernate.connection.datasource) is being used for identifying the database.
 
Any idea how to make tomcat reconnect to the database with out any problems ?
 
Any help is appreciated.
 
Thanks
Kishore Kolli
Software Engineer
Siemens
 

15:38:01,666 ERROR [JDBCTransaction] Begin failed
org.postgresql.util.PSQLException: An I/O error has occured while flushing the output - Exception: java.net.SocketException: Connection reset by peer: socket write error
Stack Trace:
 
java.net.SocketException: Connection reset by peer: socket write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
 at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
 at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
 at org.postgresql.core.PGStream.flush(PGStream.java:412)
 at org.postgresql.core.QueryExecutor.sendQueryV3(QueryExecutor.java:339)
 at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:122)
 at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:101)
 at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
 at org.postgresql.jdbc1.AbstractJdbc1Connection.execSQL(AbstractJdbc1Connection.java:867)
 at org.postgresql.jdbc1.AbstractJdbc1Connection.setAutoCommit(AbstractJdbc1Connection.java:1258)
 at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:266)
 at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:338)
 at net.sf.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:44)
 at net.sf.hibernate.transaction.JDBCTransactionFactory.beginTransaction(JDBCTransactionFactory.java:19)
 at net.sf.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:2158)
 at com.siemens.sn.phoenix.db.DbSession.beginDbTransaction(DbSession.java:291)
 at com.siemens.sn.phoenix.db.DbSession.beginTransaction(DbSession.java:282)
 at com.siemens.sn.phoenix.bl.ObjectMgr.getDomainObjects(ObjectMgr.java:49)
 at com.siemens.sn.phoenix.ui.NpmBasePage.getObjectList(NpmBasePage.java:1657)
 at com.siemens.sn.phoenix.ui.NpmBasePage.getObjectList(NpmBasePage.java:1579)
 at org.apache.jsp.base_jsp._jspService(base_jsp.java:253)
 at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
 at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
 at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
 at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:193)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:781)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:549)
 at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:589)
 at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
 at java.lang.Thread.run(Thread.java:595)
End of Stack Trace
 
 at org.postgresql.core.PGStream.flush(PGStream.java:416)
 at org.postgresql.core.QueryExecutor.sendQueryV3(QueryExecutor.java:339)
 at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:122)
 at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:101)
 at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
 at org.postgresql.jdbc1.AbstractJdbc1Connection.execSQL(AbstractJdbc1Connection.java:867)
 at org.postgresql.jdbc1.AbstractJdbc1Connection.setAutoCommit(AbstractJdbc1Connection.java:1258)
 at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:266)
 at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:338)
 at net.sf.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:44)
 at net.sf.hibernate.transaction.JDBCTransactionFactory.beginTransaction(JDBCTransactionFactory.java:19)
 at net.sf.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:2158)
 at com.siemens.sn.phoenix.db.DbSession.beginDbTransaction(DbSession.java:291)
 at com.siemens.sn.phoenix.db.DbSession.beginTransaction(DbSession.java:282)
 at com.siemens.sn.phoenix.bl.ObjectMgr.getDomainObjects(ObjectMgr.java:49)
 at com.siemens.sn.phoenix.ui.NpmBasePage.getObjectList(NpmBasePage.java:1657)
 at com.siemens.sn.phoenix.ui.NpmBasePage.getObjectList(NpmBasePage.java:1579)
 at org.apache.jsp.base_jsp._jspService(base_jsp.java:253)
 at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
 at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
 at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
 at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:193)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:781)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:549)
 at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:589)
 at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
 at java.lang.Thread.run(Thread.java:595)
On Dec 8, 2005, at 4:58 PM, Kishore Kolli wrote:


<excerpt><fontfamily><param>Arial</param><smaller> The application
dumps the following error in the following cases:</smaller></fontfamily>

<fontfamily><param>Arial</param><smaller>   1. The postgres process is
restarted.</smaller></fontfamily>

<fontfamily><param>Arial</param><smaller>   2. The network connection
between the machine running postgres and tomcat has been reset.</smaller></fontfamily>

[...] 

<fontfamily><param>Arial</param><smaller>Any idea how to make tomcat
reconnect to the database with out any problems ?</smaller></fontfamily>

 

</excerpt>

It's not a Postgres specific answer, but here's an answer nonetheless:


It appears from the stack trace that you are using the Jakarta Commons
DBCP package.  If you set a validationQuery on your connection pool,
the pool manager will automatically test your database connections
before giving them to Hibernate.  If you have had a network outage or
server restart, the pooled connections will fail that test, and the
DBCP will dump those broken connections and open new ones before
returning them to Hibernate.


For your validation, pick a simple query that is guaranteed to return
at least one row, but will not be expensive to execute (because it
will be executed frequently!).  For example, you could use "SELECT
now()", which doesn't even require disk access.


See http://jakarta.apache.org/commons/dbcp/configuration.html for
details on DBCP configuration.


-JonathanOn Dec 8, 2005, at 4:58 PM, Kishore Kolli wrote:

>  The application dumps the following error in the following cases:
>    1. The postgres process is restarted.
>    2. The network connection between the machine running postgres and
> tomcat has been reset.
> [...] 
> Any idea how to make tomcat reconnect to the database with out any
> problems ?
>  

It's not a Postgres specific answer, but here's an answer nonetheless:

It appears from the stack trace that you are using the Jakarta Commons
DBCP package.  If you set a validationQuery on your connection pool,
the pool manager will automatically test your database connections
before giving them to Hibernate.  If you have had a network outage or
server restart, the pooled connections will fail that test, and the
DBCP will dump those broken connections and open new ones before
returning them to Hibernate.

For your validation, pick a simple query that is guaranteed to return
at least one row, but will not be expensive to execute (because it will
be executed frequently!).  For example, you could use "SELECT now()",
which doesn't even require disk access.

See http://jakarta.apache.org/commons/dbcp/configuration.html for
details on DBCP configuration.

-Jonathan
Thanks a bunch.. That worked.

-kishore

On 12/9/05, Jonathan Fuerth <fuerth@sqlpower.ca> wrote:
On Dec 8, 2005, at 4:58 PM, Kishore Kolli wrote:

> The application dumps the following error in the following cases:
> 1. The postgres process is restarted.
> 2. The network connection between the machine running postgres and
> tomcat has been reset.
> [...]
> Any idea how to make tomcat reconnect to the database with out any
> problems ?
>

It's not a Postgres specific answer, but here's an answer nonetheless:

It appears from the stack trace that you are using the Jakarta Commons
DBCP package.  If you set a validationQuery on your connection pool,
the pool manager will automatically test your database connections
before giving them to Hibernate.  If you have had a network outage or
server restart, the pooled connections will fail that test, and the
DBCP will dump those broken connections and open new ones before
returning them to Hibernate.

For your validation, pick a simple query that is guaranteed to return
at least one row, but will not be expensive to execute (because it will
be executed frequently!).  For example, you could use "SELECT now()",
which doesn't even require disk access.

See http://jakarta.apache.org/commons/dbcp/configuration.html for
details on DBCP configuration.

-Jonathan