You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Keith Wall (JIRA)" <ji...@apache.org> on 2015/10/27 17:34:27 UTC

[jira] [Created] (QPID-6813) Consumer tasks may run on the Broker thread leading to deadlock

Keith Wall created QPID-6813:
--------------------------------

             Summary: Consumer tasks may run on the Broker thread leading to deadlock
                 Key: QPID-6813
                 URL: https://issues.apache.org/jira/browse/QPID-6813
             Project: Qpid
          Issue Type: Bug
          Components: Java Broker
    Affects Versions: qpid-java-6.0
            Reporter: Keith Wall
            Assignee: Keith Wall
             Fix For: qpid-java-6.0


Since QPID-6597, a consumer's tasks may run on the Broker's task executor rather than the Virtualhost's as was intended.  This may lead to deadlock.

The problem occurs because ACO discovers its task executor from its parents.  If a ACO has multiple parents, the chosen parent is non-deterministic.  This didn't used to matter because Consumers parents Queue and Session both shared a Virtualhost ancestor, so the Virtualhost task executor was always selector.  However, QPID-6597, re-homed Connection (the parent of Session) to be a child of Broker.  Since that point the executor used has been non-deterministic.

Testing with very large numbers of connections exposed a deadlock between the Broker and Virtualhost configuration thread.

{code:java}
protected AbstractConfiguredObject(final Map<Class<? extends ConfiguredObject>, ConfiguredObject<?>> parents,
                                       Map<String, Object> attributes)
    {
        this(parents, attributes, parents.values().iterator().next().getChildExecutor());
    }
{code}

{noformat}
"VirtualHostNode-default-Config" prio=10 tid=0x00007f48701a0000 nid=0xd714 waiting on condition [0x00007f48ac728000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000e7ecd4a8> (a com.google.common.util.concurrent.AbstractFuture$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
        at org.apache.qpid.server.model.AbstractConfiguredObject.doSync(AbstractConfiguredObject.java:1937)
        at org.apache.qpid.server.model.AbstractConfiguredObject.open(AbstractConfiguredObject.java:452)
        at org.apache.qpid.server.queue.QueueConsumerImpl.<init>(QueueConsumerImpl.java:136)
        at org.apache.qpid.server.queue.AbstractQueue.addConsumerInternal(AbstractQueue.java:864)
        at org.apache.qpid.server.queue.AbstractQueue.access$200(AbstractQueue.java:119)
        at org.apache.qpid.server.queue.AbstractQueue$6.execute(AbstractQueue.java:695)
        at org.apache.qpid.server.queue.AbstractQueue$6.execute(AbstractQueue.java:691)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
 
   Locked ownable synchronizers:
        - <0x00000000e5953fb0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
 
"Broker-Config" prio=10 tid=0x00007f49c8f67000 nid=0xd70d waiting on condition [0x00007f48ac95e000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ea9a2940> (a com.google.common.util.concurrent.AbstractFuture$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
        at org.apache.qpid.server.model.AbstractConfiguredObject.doSync(AbstractConfiguredObject.java:1937)
        at org.apache.qpid.server.virtualhost.AbstractVirtualHost.removeQueue(AbstractVirtualHost.java:734)
        at org.apache.qpid.server.queue.AbstractQueue$7.run(AbstractQueue.java:971)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.qpid.server.queue.AbstractQueue.unregisterConsumer(AbstractQueue.java:966)
        at org.apache.qpid.server.queue.QueueConsumerImpl.onClose(QueueConsumerImpl.java:282)
        at org.apache.qpid.server.model.AbstractConfiguredObject$5$1.run(AbstractConfiguredObject.java:666)
        at org.apache.qpid.server.model.AbstractConfiguredObject$19.onSuccess(AbstractConfiguredObject.java:2145)
        at com.google.common.util.concurrent.Futures$6.run(Futures.java:1319)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:422)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:172)
        at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
        at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
        at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:170)
        at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1322)
        at org.apache.qpid.server.model.AbstractConfiguredObject.doAfter(AbstractConfiguredObject.java:2138)
        at org.apache.qpid.server.model.AbstractConfiguredObject$ChainedSettableFuture.then(AbstractConfiguredObject.java:2200)
        at org.apache.qpid.server.model.AbstractConfiguredObject$5.execute(AbstractConfiguredObject.java:654)
        at org.apache.qpid.server.model.AbstractConfiguredObject$5.execute(AbstractConfiguredObject.java:645)
        at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:520)
        at org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:513)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
 
   Locked ownable synchronizers:
        - <0x00000000e59603c8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
        - <0x00000000ea413e18> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org