Re: Connection terminated by the server causes deadlock in jdbc client side connection - Mailing list pgsql-jdbc

From Leonardo Frittelli
Subject Re: Connection terminated by the server causes deadlock in jdbc client side connection
Date
Msg-id CACh06N3b6h+VPqRPTCj8XuodMLuftMCOp=gHQNhjEMaq3th68g@mail.gmail.com
Whole thread Raw
In response to Re: Connection terminated by the server causes deadlock in jdbc client side connection  ("Steffen Heil (Mailinglisten)" <lists@steffen-heil.de>)
Responses Re: Connection terminated by the server causes deadlock in jdbc client side connection
List pgsql-jdbc
Steffen,

I can't really comment on what is the behavior of the PG server, but at least in the scenario that I am having (replication taking priority over a local query in hot_standby) either the client does not appear to receive an RST or Java 1.7 is still trying to write to that socket even if the reading end is gone, or even worse: the server is not actually closing it but simply no longer reading (Perhaps this is indeed hinting at a bug in the server side too?)

The stack trace does not give much detail either. This is the new trace I get after patching the driver to use abort() instead of close():

...
        at java.lang.Thread.run(Unknown Source)
Caused by: org.postgresql.util.PSQLException: FATAL: terminating connection due to conflict with recovery
  Detail: User query might have needed to see row versions that must be removed.
  Hint: In a moment you should be able to reconnect to the database and repeat your command.
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:562)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:420)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:305)
        at com.jolbox.bonecp.PreparedStatementHandle.executeQuery(PreparedStatementHandle.java:174)
        at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:56)
        ... 42 more

As you can see, the IO Exception which caused the PSQL Exception is not coming up in the stack trace. 

Just for your awareness, please note that we are now using a snapshot that Dave created for us with the change above mentioned (to use abort() instead of close()) and we no longer see the issue now.

Regards,

Leonardo



On 8 October 2015 at 13:46, Steffen Heil (Mailinglisten) <lists@steffen-heil.de> wrote:
Hi


> I was referring to the fact that the TCP connection is indeed left 'half
> open'
> with the jdbc client side still assuming it's 'properly' open and thus
> trying to
> send a graceful close command over the socket ("X" in v2 or "X4" in v3) . It
> is
> this action that indefinitely blocks the client thread, which is then no
> longer
> usable by the application.

But it should not block indefinitely. Why would it?
If the server closed the connection, it should reply with a RST and the client
should detect that and throw an IOException( "Socket closed" ).


> All of this said, I still think it is conceptually incorrect to attempt a
> graceful
> closure after an IO error in the client.

Agreed. IOExceptions on blocking sockets usually indicate that you cannot do
anything good with them anymore.


Regards,
  Steffen



pgsql-jdbc by date:

Previous
From: "Steffen Heil (Mailinglisten)"
Date:
Subject: Re: Connection terminated by the server causes deadlock in jdbc client side connection
Next
From: Dave Cramer
Date:
Subject: Release 1204 released