You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Alex Rudyy (JIRA)" <ji...@apache.org> on 2013/03/20 12:49:15 UTC

[jira] [Created] (QPID-4660) [Java Broker] A deadlock can sporadically occur on broker shutdown when stopping a Jetty server which is in the process of sending a response over HTTPS

Alex Rudyy created QPID-4660:
--------------------------------

             Summary: [Java Broker] A deadlock can sporadically occur on broker shutdown when stopping a Jetty server which is in the process of sending a response over HTTPS
                 Key: QPID-4660
                 URL: https://issues.apache.org/jira/browse/QPID-4660
             Project: Qpid
          Issue Type: Bug
          Components: Java Broker
    Affects Versions: 0.21
         Environment: RHEL 5.3, JDK 1.7.0_14
            Reporter: Alex Rudyy


When running a system test suite org.apache.qpid.systest.rest.BasicAuthRestTest with JDK 1.7 a deadlock might occur on shutting down the broker whilst it still tries to send some data over HTTPS:

{noformat}
Found one Java-level deadlock:
=============================
"qtp538763660-62":
  waiting to lock monitor 0x000000000c91c440 (object 0x00000000f9404e28, a sun.security.ssl.SSLSocketImpl),
  which is held by "Broker-Configuration-Thread"
"Broker-Configuration-Thread":
  waiting for ownable synchronizer 0x00000000f95c6278, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
  which is held by "qtp538763660-62"

Java stack information for the threads listed above:
===================================================
"qtp538763660-62":
        at sun.security.ssl.SSLSocketImpl.getConnectionState(SSLSocketImpl.java:649)
        - waiting to lock <0x00000000f9404e28> (a sun.security.ssl.SSLSocketImpl)
        at sun.security.ssl.SSLSocketImpl.isClosed(SSLSocketImpl.java:1446)
        at java.net.Socket.getTcpNoDelay(Socket.java:953)
        at sun.security.ssl.BaseSSLSocketImpl.getTcpNoDelay(BaseSSLSocketImpl.java:345)
        at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:819)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:801)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
        - locked <0x00000000f95c60b8> (a sun.security.ssl.AppOutputStream)
        at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:354)
        at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:159)
        at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:189)
        at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:876)
        at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:438)
        at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:94)
        at org.eclipse.jetty.server.AbstractHttpConnection$Output.flush(AbstractHttpConnection.java:1016)
        at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:173)
        at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101)
        at org.eclipse.jetty.util.ByteArrayISO8859Writer.writeTo(ByteArrayISO8859Writer.java:102)
        at org.eclipse.jetty.server.Response.sendError(Response.java:374)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:562)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
        at org.eclipse.jetty.server.Server.handle(Server.java:348)
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
        at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
        at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:884)
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:938)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:630)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
        at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
        at java.lang.Thread.run(Thread.java:722)
"Broker-Configuration-Thread":
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000f95c6278> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
        at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
        at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:799)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:672)
        at sun.security.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:2007)
        at sun.security.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1834)
        at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1602)
        - locked <0x00000000f9404e28> (a sun.security.ssl.SSLSocketImpl)
        at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1538)
        at org.eclipse.jetty.io.bio.SocketEndPoint.close(SocketEndPoint.java:156)
        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.close(SocketConnector.java:233)
        at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.close(SslSocketConnector.java:613)
        at org.eclipse.jetty.server.bio.SocketConnector.doStop(SocketConnector.java:167)
        at org.eclipse.jetty.server.ssl.SslSocketConnector.doStop(SslSocketConnector.java:371)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:84)
        - locked <0x00000000f94069a0> (a java.lang.Object)
        at org.eclipse.jetty.server.Server.doStop(Server.java:318)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:84)
        - locked <0x00000000f9406dc0> (a java.lang.Object)
        at org.apache.qpid.server.management.plugin.HttpManagement.stop(HttpManagement.java:187)
        at org.apache.qpid.server.management.plugin.HttpManagement.setState(HttpManagement.java:156)
        at org.apache.qpid.server.model.adapter.AbstractAdapter.setDesiredState(AbstractAdapter.java:88)
        at org.apache.qpid.server.model.adapter.BrokerAdapter.changeState(BrokerAdapter.java:776)
        - locked <0x00000000f958c040> (a java.util.HashMap)
        at org.apache.qpid.server.model.adapter.BrokerAdapter.setState(BrokerAdapter.java:749)
        at org.apache.qpid.server.model.adapter.AbstractAdapter.setDesiredState(AbstractAdapter.java:88)
        at org.apache.qpid.server.configuration.updater.ChangeStateTask.call(ChangeStateTask.java:59)
        at org.apache.qpid.server.configuration.updater.ChangeStateTask.call(ChangeStateTask.java:28)
        at org.apache.qpid.server.configuration.updater.TaskExecutor.executeTask(TaskExecutor.java:213)
        at org.apache.qpid.server.configuration.updater.TaskExecutor.access$200(TaskExecutor.java:46)
        at org.apache.qpid.server.configuration.updater.TaskExecutor$CallableWrapper$1.run(TaskExecutor.java:252)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.qpid.server.configuration.updater.TaskExecutor$CallableWrapper.call(TaskExecutor.java:247)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)

Found 1 deadlock.

{noformat}

It seems that the issue does not exist on JDK 1.6. I've been running tests for 2 days with JDK 1.6.0_33 but was not able to reproduce it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org