You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2013/08/14 02:05:47 UTC
[jira] [Commented] (SOLR-5145) wait w/o notify situation happening
on shutdown in CoreContainer/CloserThread/SolrCores.modifyLock
[ https://issues.apache.org/jira/browse/SOLR-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13739049#comment-13739049 ]
Hoss Man commented on SOLR-5145:
--------------------------------
Thread dump below
[~markrmiller@gmail.com], [~romseygeek], [~erickerickson] ... any ideas here?
{noformat}
2013-08-13 16:47:47
Full thread dump OpenJDK 64-Bit Server VM (20.0-b12 mixed mode):
"Attach Listener" daemon prio=10 tid=0x00007f7a04001000 nid=0x4e01 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Thread-8" prio=10 tid=0x00007f79ec18c000 nid=0x208c in Object.wait() [0x00007f7a0d2a7000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e068d718> (a java.lang.Object)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:961)
- locked <0x00000000e068d718> (a java.lang.Object)
"searcherExecutor-46-thread-1" prio=10 tid=0x00007f79e400d800 nid=0x208b waiting on condition [0x00007f7a0d0a5000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f66ff860> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1069)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1131)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
"TEST-TestManagedSchema.testPersistUniqueKey-seed#[BAF0D5A2ABB1707B]" prio=10 tid=0x00007f79e80ba800 nid=0x1fb0 in Object.wait() [0x00007f7a0ea9d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000f690b880> (a org.apache.solr.core.CloserThread)
at java.lang.Thread.join(Thread.java:1203)
- locked <0x00000000f690b880> (a org.apache.solr.core.CloserThread)
at java.lang.Thread.join(Thread.java:1256)
at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:358)
at org.apache.solr.util.TestHarness.close(TestHarness.java:338)
at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:536)
at org.apache.solr.schema.TestManagedSchema.testPersistUniqueKey(TestManagedSchema.java:402)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:679)
"SUITE-TestManagedSchema-seed#[BAF0D5A2ABB1707B]" prio=10 tid=0x00007f7a282ee800 nid=0x1fae in Object.wait() [0x00007f7a0ece5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e0000090> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1211)
- locked <0x00000000e0000090> (a java.lang.Thread)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:787)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:385)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:556)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:492)
"Low Memory Detector" daemon prio=10 tid=0x00007f7a280d1800 nid=0x1faa runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0x00007f7a280cf800 nid=0x1fa9 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x00007f7a280cc800 nid=0x1fa8 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x00007f7a280ca000 nid=0x1fa7 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x00007f7a2807d800 nid=0x1f9c in Object.wait() [0x00007f7a0fefd000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e0002510> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
- locked <0x00000000e0002510> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
"Reference Handler" daemon prio=10 tid=0x00007f7a2807b800 nid=0x1f9b in Object.wait() [0x00007f7a0fffe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e0000168> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x00000000e0000168> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x00007f7a2800d000 nid=0x1f90 in Object.wait() [0x00007f7a2f16a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e00020c8> (a com.carrotsearch.randomizedtesting.RandomizedRunner$1)
at java.lang.Thread.join(Thread.java:1203)
- locked <0x00000000e00020c8> (a com.carrotsearch.randomizedtesting.RandomizedRunner$1)
at java.lang.Thread.join(Thread.java:1256)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:502)
at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:399)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:180)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:275)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
"VM Thread" prio=10 tid=0x00007f7a28077000 nid=0x1f9a runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f7a28018000 nid=0x1f94 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f7a2801a000 nid=0x1f96 runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f7a2801c000 nid=0x1f97 runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f7a2801d800 nid=0x1f98 runnable
"VM Periodic Task Thread" prio=10 tid=0x00007f7a280d4000 nid=0x1fab waiting on condition
JNI global references: 1522
{noformat}
> wait w/o notify situation happening on shutdown in CoreContainer/CloserThread/SolrCores.modifyLock
> --------------------------------------------------------------------------------------------------
>
> Key: SOLR-5145
> URL: https://issues.apache.org/jira/browse/SOLR-5145
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
>
> Working on SOLR-4952 i was running tests on the 4x branch and noticed TestManagedSchema.testPersistUniqueKey stalled for 1623s -- but nothing baout hte situation seems to be specific to that test.
> I'll attache a threaddump, but the key things i note...
> * TestHarness is trying to shutdown the CoreContainer
> * CoreContainer.shutdown(CoreContainer.java:358) is waiting on <0x00000000f690b880> (a org.apache.solr.core.CloserThread)
> * CloserThread.run(CoreContainer.java:961) is waiting on <0x00000000e068d718> (a java.lang.Object)
> ** that's aparently "SolrCores.modifyLock"
> * a "searcherExecutor-46-thread-1" is still alive and parking to wait for <0x00000000f66ff860> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> * no other threads seem to be doing anything -- so as far as i can tell nothing is ever going to notify on that modifyLock
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org