You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Graham Leggett <mi...@sharp.fm> on 2013/10/04 11:53:50 UTC
amqp client: "Connection has been shutdown" triggers stuck thread
Hi all,
I have a client that makes once off connections to an activemq server over aqmp to poll for messages from time to time (no long lived connections are used). This works great for a period of time, until we suddenly get the unexpected exception below printed to the console. At this point the qpid aqmp client thread hangs, and our application node becomes stuck.
From what I can see, this tends to happen when it takes use a long time to consume a message (we are doing heavy processing on the message when it arrives, and may take many tens of seconds, even minutes).
We are using the trunk version of qpid (to work around another bug).
Does this look familiar to anyone?
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:103)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.qpid.amqp_1_0.client.Connection.doRead(Connection.java:370)
at org.apache.qpid.amqp_1_0.client.Connection.access$000(Connection.java:42)
at org.apache.qpid.amqp_1_0.client.Connection$2.run(Connection.java:244)
at java.lang.Thread.run(Thread.java:724)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1844)
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1808)
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1753)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:127)
at org.apache.qpid.amqp_1_0.framing.ConnectionHandler$BytesOutputHandler.processBytes(ConnectionHandler.java:418)
at org.apache.qpid.amqp_1_0.framing.ConnectionHandler$HeaderBytesSource.getBytes(ConnectionHandler.java:336)
at org.apache.qpid.amqp_1_0.framing.ConnectionHandler$SequentialBytesSource.getBytes(ConnectionHandler.java:371)
at org.apache.qpid.amqp_1_0.framing.ConnectionHandler$BytesOutputHandler.run(ConnectionHandler.java:404)
... 1 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.read(InputRecord.java:480)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
... 5 more
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.read(InputRecord.java:480)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.qpid.amqp_1_0.client.Connection.doRead(Connection.java:370)
at org.apache.qpid.amqp_1_0.client.Connection.access$000(Connection.java:42)
at org.apache.qpid.amqp_1_0.client.Connection$2.run(Connection.java:244)
at java.lang.Thread.run(Thread.java:724)
Regards,
Graham
--
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: amqp client: "Connection has been shutdown" triggers stuck thread
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Graham,
can't say I've seen precisely that scenario before... There seem to be two
different issues at play here, firstly the SSL connection shutting down,
and secondly the fact that this is causing the JMS client to hang. The
second would definitely seem to be an issue in our code... Do you have a
repeatable test case that can reproduce the issue and/or a thread dump from
the JMS client when it is hung? If you can raise a JIRA (
https://issues.apache.org/jira/secure/CreateIssue!default.jspa?selectedProjectId=10450)
and attach any such information it would greatly assist us in trying
to
fix this issue.
As to why the SSL connection is being closed in the first place... is
there any information from ActiveMQ on that as it would seem that it is
that which is closing the connection?
Cheers,
Rob
On 4 October 2013 11:53, Graham Leggett <mi...@sharp.fm> wrote:
> Hi all,
>
> I have a client that makes once off connections to an activemq server over
> aqmp to poll for messages from time to time (no long lived connections are
> used). This works great for a period of time, until we suddenly get the
> unexpected exception below printed to the console. At this point the qpid
> aqmp client thread hangs, and our application node becomes stuck.
>
> From what I can see, this tends to happen when it takes use a long time to
> consume a message (we are doing heavy processing on the message when it
> arrives, and may take many tens of seconds, even minutes).
>
> We are using the trunk version of qpid (to work around another bug).
>
> Does this look familiar to anyone?
>
> javax.net.ssl.SSLException: Connection has been shutdown:
> javax.net.ssl.SSLException: java.net.SocketException: Connection reset
> at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476)
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:103)
> at java.io.InputStream.read(InputStream.java:101)
> at
> org.apache.qpid.amqp_1_0.client.Connection.doRead(Connection.java:370)
> at
> org.apache.qpid.amqp_1_0.client.Connection.access$000(Connection.java:42)
> at
> org.apache.qpid.amqp_1_0.client.Connection$2.run(Connection.java:244)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: javax.net.ssl.SSLException: java.net.SocketException:
> Connection reset
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1844)
> at
> sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1808)
> at
> sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1753)
> at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:127)
> at
> org.apache.qpid.amqp_1_0.framing.ConnectionHandler$BytesOutputHandler.processBytes(ConnectionHandler.java:418)
> at
> org.apache.qpid.amqp_1_0.framing.ConnectionHandler$HeaderBytesSource.getBytes(ConnectionHandler.java:336)
> at
> org.apache.qpid.amqp_1_0.framing.ConnectionHandler$SequentialBytesSource.getBytes(ConnectionHandler.java:371)
> at
> org.apache.qpid.amqp_1_0.framing.ConnectionHandler$BytesOutputHandler.run(ConnectionHandler.java:404)
> ... 1 more
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:189)
> at java.net.SocketInputStream.read(SocketInputStream.java:121)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
> at sun.security.ssl.InputRecord.read(InputRecord.java:480)
> at
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
> at
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
> at
> sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702)
> at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
> ... 5 more
> java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:189)
> at java.net.SocketInputStream.read(SocketInputStream.java:121)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
> at sun.security.ssl.InputRecord.read(InputRecord.java:480)
> at
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
> at
> sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
> at java.io.InputStream.read(InputStream.java:101)
> at
> org.apache.qpid.amqp_1_0.client.Connection.doRead(Connection.java:370)
> at
> org.apache.qpid.amqp_1_0.client.Connection.access$000(Connection.java:42)
> at
> org.apache.qpid.amqp_1_0.client.Connection$2.run(Connection.java:244)
> at java.lang.Thread.run(Thread.java:724)
>
> Regards,
> Graham
> --
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>