Thread: java.net.SocketException: Broken pipe

java.net.SocketException: Broken pipe

From
Phil.Hourihane@meridianp2p.com
Date:

Hello, World.

I have an intermittent problem that I am having trouble with, using postgresql
7.3.4. It manifests itself as a SocketException...

java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:69)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:127)
        at org.postgresql.PG_Stream.flush(PG_Stream.java:352)
        at org.postgresql.core.QueryExecutor.sendQuery(QueryExecutor.java:159)
        at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:70)
        ...etc...

... but it doesn't seem to correspond to the mails on this list that mention
this trace. The previous mails warn that this can happen if an earlier statement
includes a null character (char 0). I tried to reproduce the problem that
way but instead I was prevented by...

java.lang.IllegalArgumentException: \0 not allowed
        at org.postgresql.jdbc1.AbstractJdbc1Statement.escapeString(AbstractJdbc1Statement.java:955)
        at org.postgresql.jdbc1.AbstractJdbc1Statement.setString(AbstractJdbc1Statement.java:917)
        at org.postgresql.jdbc1.AbstractJdbc1Statement.setString(AbstractJdbc1Statement.java:900)
        ...etc...

Now, the problem seems to occur only intermittently, and will 'repair' itself
with no action on my part. It does however, seem to occur in clumps - it will
fail a few times in a short space of time before working again.

I have created a statement at boot time for this purpose (it is a statement
that gets used a lot, so I prefer to cache it), rather than create a new
statement for each invocation. Some posts on the mailing list mention this
as a problem (the backend shutting down the connection), but I have no idea
how to control this behaviour.

I don't seen to be able to reproduce this behaviour on demand; I have seen
it in logs of a system that is running long-term, not just a development box
that is working for a little bit at a time.

It doesn't seem to be related to heavy loads on the system either - the load
has been pretty light when it occurred.

Has anybody else encountered this problem or had any success in dealing with it?

PHiL

Re: java.net.SocketException: Broken pipe

From
Barry Lind
Date:
Phil,

A possible cause is a firewall or some other device between the client
and the server that drops connections after a certain time.  (Most
firewall don't allow you to keep connections open indefinitely).

thanks,
--Barry


Phil.Hourihane@meridianp2p.com wrote:
> Hello, World.
>
> I have an intermittent problem that I am having trouble with, using
> postgresql
> 7.3.4. It manifests itself as a SocketException...
>
> java.net.SocketException: Broken pipe
>         at java.net.SocketOutputStream.socketWrite0(Native Method)
>         at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>         at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:69)
>         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:127)
>         at org.postgresql.PG_Stream.flush(PG_Stream.java:352)
>         at
> org.postgresql.core.QueryExecutor.sendQuery(QueryExecutor.java:159)
>         at
> org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:70)
>         ...etc...
>
> ... but it doesn't seem to correspond to the mails on this list that
> mention
> this trace. The previous mails warn that this can happen if an earlier
> statement
> includes a null character (char 0). I tried to reproduce the problem that
> way but instead I was prevented by...
>
> java.lang.IllegalArgumentException: \0 not allowed
>         at
> org.postgresql.jdbc1.AbstractJdbc1Statement.escapeString(AbstractJdbc1Statement.java:955)
>         at
> org.postgresql.jdbc1.AbstractJdbc1Statement.setString(AbstractJdbc1Statement.java:917)
>         at
> org.postgresql.jdbc1.AbstractJdbc1Statement.setString(AbstractJdbc1Statement.java:900)
>         ...etc...
>
> Now, the problem seems to occur only intermittently, and will 'repair'
> itself
> with no action on my part. It does however, seem to occur in clumps - it
> will
> fail a few times in a short space of time before working again.
>
> I have created a statement at boot time for this purpose (it is a
> statement
> that gets used a lot, so I prefer to cache it), rather than create a new
> statement for each invocation. Some posts on the mailing list mention this
> as a problem (the backend shutting down the connection), but I have no
> idea
> how to control this behaviour.
>
> I don't seen to be able to reproduce this behaviour on demand; I have seen
> it in logs of a system that is running long-term, not just a development
> box
> that is working for a little bit at a time.
>
> It doesn't seem to be related to heavy loads on the system either - the
> load
> has been pretty light when it occurred.
>
> Has anybody else encountered this problem or had any success in dealing
> with it?
>
> PHiL




Re: java.net.SocketException: Broken pipe

From
Barry Lind
Date:
Phil,

Phil.Hourihane@meridianp2p.com wrote:
>>Phil,
>>
>>A possible cause is a firewall or some other device between the client
>>and the server that drops connections after a certain time.  (Most
>>firewall don't allow you to keep connections open indefinitely).
>>
>>thanks,
>>--Barry
>
>
> Thanks, Barrry, for your reply.
>
> Unfortunately, I don't think this is my problem.There is no firewall in
> place which would prevent a connection from lasting for any
> length of time. In fact, the statement that I am caching is built
> around a connection pool, so that the statement should not
> actually be using the same connection each time it is run at all.
>

Here is your problem.  A statement object is owned by a connection.  You
cannot cache statement objects across different connections.  What will
actually happen is if you later use the statement, it will still be
using the connection that created it, even if you have returned that
connection back to the pool.  What I suspect is happening is that your
connection pool is occasionally closing connections and it just so
happens that the connection closed is the one used by your cached
statement.  If you want to cache a statement like this, you will also
need to cache the connection that goes with that statement.

thanks,
--Barry




Re: java.net.SocketException: Broken pipe

From
Phil.Hourihane@meridianp2p.com
Date:

Thanks for the reply Barry.

I appreciate the thought you've put into this, and I'm going to switch out the caching
and retain the use of the pool (rather than keep the caching and not use the pool
for this problem).

So, thanks for your help

PHiL