You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Keith Wall (JIRA)" <ji...@apache.org> on 2015/10/28 19:39:27 UTC
[jira] [Created] (QPID-6814) IO Selector deadlock
Keith Wall created QPID-6814:
--------------------------------
Summary: IO Selector deadlock
Key: QPID-6814
URL: https://issues.apache.org/jira/browse/QPID-6814
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: qpid-java-6.0
Environment: Darwin 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64
Reporter: Keith Wall
Fix For: qpid-java-6.0
Testing with trunk this evening with perftests client, I encountered the following deadlock. It appears that we allow a channel to become registered concurrently on two separate selectors.
{noformat}
Found one Java-level deadlock:
=============================
"IO-/127.0.0.1:56913":
waiting to lock monitor 0x00007f9754aa2ad8 (object 0x0000000780448270, a java.util.HashSet),
which is held by "Selector-virtualhost-default-iopool"
"Selector-virtualhost-default-iopool":
waiting to lock monitor 0x00007f9755964bd8 (object 0x00000007871fce50, a java.lang.Object),
which is held by "IO-/127.0.0.1:56916"
"IO-/127.0.0.1:56916":
waiting to lock monitor 0x00007f97549a0718 (object 0x00000007804373f8, a java.util.HashSet),
which is held by "Selector-virtualhost-default-iopool"
"Selector-virtualhost-default-iopool":
waiting to lock monitor 0x00007f9755b53c88 (object 0x00000007872865b0, a java.lang.Object),
which is held by "IO-/127.0.0.1:56908"
"IO-/127.0.0.1:56908":
waiting to lock monitor 0x00007f975453c7c8 (object 0x0000000780435f18, a java.util.HashSet),
which is held by "Selector-virtualhost-default-iopool"
"Selector-virtualhost-default-iopool":
waiting to lock monitor 0x00007f9755964bd8 (object 0x00000007871fce50, a java.lang.Object),
which is held by "IO-/127.0.0.1:56916"
Java stack information for the threads listed above:
===================================================
"IO-/127.0.0.1:56913":
at java.nio.channels.spi.AbstractSelector.cancel(AbstractSelector.java:89)
- waiting to lock <0x0000000780448270> (a java.util.HashSet)
at java.nio.channels.spi.AbstractSelectionKey.cancel(AbstractSelectionKey.java:73)
- locked <0x000000078720b7e8> (a sun.nio.ch.SelectionKeyImpl)
at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:237)
- locked <0x000000078720b810> (a java.lang.Object)
at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
- locked <0x00000007871f7660> (a java.lang.Object)
at org.apache.qpid.server.transport.NonBlockingConnection.shutdown(NonBlockingConnection.java:288)
at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:268)
at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:111)
at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:437)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:293)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
"Selector-virtualhost-default-iopool":
at java.nio.channels.spi.AbstractSelectableChannel.removeKey(AbstractSelectableChannel.java:127)
- waiting to lock <0x00000007871fce50> (a java.lang.Object)
at java.nio.channels.spi.AbstractSelector.deregister(AbstractSelector.java:185)
at sun.nio.ch.KQueueSelectorImpl.implDereg(KQueueSelectorImpl.java:226)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:150)
- locked <0x0000000780448270> (a java.util.HashSet)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:107)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x0000000780448340> (a sun.nio.ch.Util$2)
- locked <0x0000000780448330> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000780448210> (a sun.nio.ch.KQueueSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:246)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
"IO-/127.0.0.1:56916":
at java.nio.channels.spi.AbstractSelector.cancel(AbstractSelector.java:89)
- waiting to lock <0x00000007804373f8> (a java.util.HashSet)
at java.nio.channels.spi.AbstractSelectionKey.cancel(AbstractSelectionKey.java:73)
- locked <0x0000000787577460> (a sun.nio.ch.SelectionKeyImpl)
at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:237)
- locked <0x00000007871fce50> (a java.lang.Object)
at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
- locked <0x00000007871fccd8> (a java.lang.Object)
at org.apache.qpid.server.transport.NonBlockingConnection.shutdown(NonBlockingConnection.java:288)
at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:268)
at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:111)
at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:437)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:293)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
"Selector-virtualhost-default-iopool":
at java.nio.channels.spi.AbstractSelectableChannel.removeKey(AbstractSelectableChannel.java:127)
- waiting to lock <0x00000007872865b0> (a java.lang.Object)
at java.nio.channels.spi.AbstractSelector.deregister(AbstractSelector.java:185)
at sun.nio.ch.KQueueSelectorImpl.implDereg(KQueueSelectorImpl.java:226)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:150)
- locked <0x00000007804373f8> (a java.util.HashSet)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:107)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x0000000780447058> (a sun.nio.ch.Util$2)
- locked <0x0000000780447048> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000780437398> (a sun.nio.ch.KQueueSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:246)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
"IO-/127.0.0.1:56908":
at java.nio.channels.spi.AbstractSelector.cancel(AbstractSelector.java:89)
- waiting to lock <0x0000000780435f18> (a java.util.HashSet)
at java.nio.channels.spi.AbstractSelectionKey.cancel(AbstractSelectionKey.java:73)
- locked <0x000000078720bed8> (a sun.nio.ch.SelectionKeyImpl)
at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:237)
- locked <0x00000007872865b0> (a java.lang.Object)
at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
- locked <0x000000078720be78> (a java.lang.Object)
at org.apache.qpid.server.transport.NonBlockingConnection.shutdown(NonBlockingConnection.java:288)
at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:268)
at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:111)
at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.run(SelectorThread.java:437)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:293)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
"Selector-virtualhost-default-iopool":
at java.nio.channels.spi.AbstractSelectableChannel.removeKey(AbstractSelectableChannel.java:127)
- waiting to lock <0x00000007871fce50> (a java.lang.Object)
at java.nio.channels.spi.AbstractSelector.deregister(AbstractSelector.java:185)
at sun.nio.ch.KQueueSelectorImpl.implDereg(KQueueSelectorImpl.java:226)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:150)
- locked <0x0000000780435f18> (a java.util.HashSet)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:100)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x0000000780435fe8> (a sun.nio.ch.Util$2)
- locked <0x0000000780435fd8> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000780435eb8> (a sun.nio.ch.KQueueSelectorImpl)
at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:242)
at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:85)
at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Found 1 deadlock.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org