You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/10/20 15:24:58 UTC

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 352 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/352/

4 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test

Error Message:
There are still nodes recoverying - waited for 320 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A:4C6547AF5C4B4E92]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:920)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1477)
	at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest

Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:     1) Thread[id=34646, name=searcherExecutor-11509-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=34681, name=searcherExecutor-11523-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=34716, name=searcherExecutor-11537-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=34495, name=searcherExecutor-11461-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=34576, name=searcherExecutor-11489-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 
   1) Thread[id=34646, name=searcherExecutor-11509-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=34681, name=searcherExecutor-11523-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=34716, name=searcherExecutor-11537-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=34495, name=searcherExecutor-11461-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=34576, name=searcherExecutor-11489-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=34646, name=searcherExecutor-11509-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=34681, name=searcherExecutor-11523-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=34716, name=searcherExecutor-11537-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=34495, name=searcherExecutor-11461-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=34576, name=searcherExecutor-11489-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=34646, name=searcherExecutor-11509-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=34681, name=searcherExecutor-11523-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=34716, name=searcherExecutor-11537-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=34495, name=searcherExecutor-11461-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=34576, name=searcherExecutor-11489-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A]:0)


FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test10Minutes

Error Message:
Core 00001_core bad! expected:<140687> but was:<134442>

Stack Trace:
java.lang.AssertionError: Core 00001_core bad! expected:<140687> but was:<134442>
	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A:D4B4ACDC6012E941]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.core.OpenCloseCoreStressTest.checkResults(OpenCloseCoreStressTest.java:280)
	at org.apache.solr.core.OpenCloseCoreStressTest.doStress(OpenCloseCoreStressTest.java:179)
	at org.apache.solr.core.OpenCloseCoreStressTest.test10Minutes(OpenCloseCoreStressTest.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 15178 lines...]
   [junit4] Suite: org.apache.solr.core.OpenCloseCoreStressTest
   [junit4]   2> 3461583 INFO  (SUITE-OpenCloseCoreStressTest-seed#[C4317875F2B7236A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/init-core-data-001
   [junit4]   2> 3461583 WARN  (SUITE-OpenCloseCoreStressTest-seed#[C4317875F2B7236A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
   [junit4]   2> 3461584 INFO  (SUITE-OpenCloseCoreStressTest-seed#[C4317875F2B7236A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3461586 INFO  (SUITE-OpenCloseCoreStressTest-seed#[C4317875F2B7236A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3461606 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test5Seconds
   [junit4]   2> 3461624 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 3461625 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3461625 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3461626 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 3461626 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@11a817c9{/solr,null,AVAILABLE}
   [junit4]   2> 3461635 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4117b554{HTTP/1.1,[http/1.1]}{127.0.0.1:37094}
   [junit4]   2> 3461635 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.e.j.s.Server Started @3461693ms
   [junit4]   2> 3461635 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37094}
   [junit4]   2> 3461636 ERROR (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3461636 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3461636 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 3461636 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 3461636 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3461636 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-20T15:06:59.736Z
   [junit4]   2> 3461637 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/solr.xml
   [junit4]   2> 3461643 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 3461736 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3461736 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3461767 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461785 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461785 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461794 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.CorePropertiesLocator Found 7 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001
   [junit4]   2> 3461795 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [00000_core, 00004_core, 00003_core, 00002_core, 00006_core, 00001_core, 00005_core]
   [junit4]   2> 3461840 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.OpenCloseCoreStressTest Starting indexing and querying
   [junit4]   2> 3461840 INFO  (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[C4317875F2B7236A]) [    ] o.a.s.c.OpenCloseCoreStressTest 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> 
   [junit4]   2> Starting a 5 second cycle, seconds left: 5. Seconds run so far: 0.
   [junit4]   2> 3461846 INFO  (Thread-56795) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189758
   [junit4]   2> 3461846 INFO  (Thread-56794) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189757
   [junit4]   2> 3461846 INFO  (Thread-56796) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189759
   [junit4]   2> 3461847 INFO  (Thread-56797) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189760
   [junit4]   2> 3461855 INFO  (Thread-56798) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189761
   [junit4]   2> 3461854 INFO  (Thread-56800) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189763
   [junit4]   2> 3461855 INFO  (Thread-56801) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189764
   [junit4]   2> 3461857 INFO  (Thread-56802) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189765
   [junit4]   2> 3461854 INFO  (Thread-56799) [    ] o.a.s.c.OneIndexer Starting indexing thread: 189762
   [junit4]   2> 3461863 INFO  (Thread-56803) [    ] o.a.s.c.OneQuery Starting query thread: 189767
   [junit4]   2> 3461863 INFO  (Thread-56804) [    ] o.a.s.c.OneQuery Starting query thread: 189768
   [junit4]   2> 3461866 INFO  (Thread-56805) [    ] o.a.s.c.OneQuery Starting query thread: 189769
   [junit4]   2> 3461867 INFO  (Thread-56808) [    ] o.a.s.c.OneQuery Starting query thread: 189771
   [junit4]   2> 3461867 INFO  (Thread-56807) [    ] o.a.s.c.OneQuery Starting query thread: 189770
   [junit4]   2> 3461869 INFO  (Thread-56809) [    ] o.a.s.c.OneQuery Starting query thread: 189772
   [junit4]   2> 3461869 INFO  (Thread-56810) [    ] o.a.s.c.OneQuery Starting query thread: 189774
   [junit4]   2> 3461884 INFO  (Thread-56812) [    ] o.a.s.c.OneQuery Starting query thread: 189776
   [junit4]   2> 3461887 INFO  (Thread-56813) [    ] o.a.s.c.OneQuery Starting query thread: 189777
   [junit4]   2> 3461894 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3461894 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3461897 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3461906 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.s.IndexSchema [00004_core] Schema name=tiny
   [junit4]   2> 3461907 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.s.IndexSchema [00000_core] Schema name=tiny
   [junit4]   2> 3461911 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3461911 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3461911 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.c.CoreContainer Creating SolrCore '00004_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00004_core, trusted=true
   [junit4]   2> 3461911 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.c.CoreContainer Creating SolrCore '00000_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00000_core, trusted=true
   [junit4]   2> 3461911 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00004_core' (registry 'solr.core.00004_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461911 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00000_core' (registry 'solr.core.00000_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461911 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.c.SolrCore [[00004_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00004_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00004_core/data/]
   [junit4]   2> 3461911 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.s.IndexSchema [00005_core] Schema name=tiny
   [junit4]   2> 3461914 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3461914 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.c.CoreContainer Creating SolrCore '00005_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00005_core, trusted=true
   [junit4]   2> 3461915 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00005_core' (registry 'solr.core.00005_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461911 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.c.SolrCore [[00000_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00000_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00000_core/data/]
   [junit4]   2> 3461915 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@509ab63b
   [junit4]   2> 3461918 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3503a836
   [junit4]   2> 3461918 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.c.SolrCore [[00005_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00005_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00005_core/data/]
   [junit4]   2> 3461921 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3461922 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7f4c51f
   [junit4]   2> 3461934 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.s.IndexSchema [00006_core] Schema name=tiny
   [junit4]   2> 3461938 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3461939 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.c.CoreContainer Creating SolrCore '00006_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00006_core, trusted=true
   [junit4]   2> 3461941 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00006_core' (registry 'solr.core.00006_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3461942 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.c.SolrCore [[00006_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00006_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00006_core/data/]
   [junit4]   2> 3461959 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@39952058
   [junit4]   2> 3461991 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3462000 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.s.IndexSchema [00002_core] Schema name=tiny
   [junit4]   2> 3462004 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3462004 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.c.CoreContainer Creating SolrCore '00002_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00002_core, trusted=true
   [junit4]   2> 3462005 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00002_core' (registry 'solr.core.00002_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3462005 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.c.SolrCore [[00002_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00002_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00002_core/data/]
   [junit4]   2> 3462009 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@22114e2d
   [junit4]   2> 3462034 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462034 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462036 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462036 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@338e4e88[00004_core] main]
   [junit4]   2> 3462037 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00004_core/conf
   [junit4]   2> 3462038 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462038 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462038 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462040 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462041 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@588e0f46[00005_core] main]
   [junit4]   2> 3462045 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id7]} 0 157
   [junit4]   2> 3462050 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00005_core/conf
   [junit4]   2> 3462056 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462058 INFO  (searcherExecutor-15692-thread-1-processing-x:00005_core) [    x:00005_core] o.a.s.c.SolrCore [00005_core] Registered new searcher Searcher@588e0f46[00005_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462049 INFO  (searcherExecutor-15690-thread-1-processing-x:00004_core) [    x:00004_core] o.a.s.c.SolrCore [00004_core] Registered new searcher Searcher@338e4e88[00004_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462049 INFO  (qtp1950205446-189783) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id8]} 0 144
   [junit4]   2> 3462058 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id3]} 0 195
   [junit4]   2> 3462059 INFO  (qtp1950205446-189789) [    x:00005_core] o.a.s.c.S.Request [00005_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=71
   [junit4]   2> 3462059 INFO  (qtp1950205446-189779) [    x:00004_core] o.a.s.c.S.Request [00004_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=82
   [junit4]   2> 3462059 INFO  (qtp1950205446-189792) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id10]} 0 0
   [junit4]   2> 3462059 INFO  (qtp1950205446-189790) [    x:00005_core] o.a.s.c.S.Request [00005_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=83
   [junit4]   2> 3462060 INFO  (qtp1950205446-189780) [    x:00005_core] o.a.s.c.S.Request [00005_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=83
   [junit4]   2> 3462060 INFO  (qtp1950205446-189724) [    x:00005_core] o.a.s.c.S.Request [00005_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=84
   [junit4]   2> 3462061 INFO  (qtp1950205446-189730) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id9]} 0 197
   [junit4]   2> 3462071 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462071 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462075 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462075 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@4565177c[00000_core] main]
   [junit4]   2> 3462077 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00000_core/conf
   [junit4]   2> 3462078 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462083 INFO  (qtp1950205446-189779) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id11]} 0 23
   [junit4]   2> 3462083 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462083 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462084 INFO  (searcherExecutor-15691-thread-1-processing-x:00000_core) [    x:00000_core] o.a.s.c.SolrCore [00000_core] Registered new searcher Searcher@4565177c[00000_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462087 INFO  (qtp1950205446-189782) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id2]} 0 183
   [junit4]   2> 3462088 INFO  (qtp1950205446-189728) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id1]} 0 224
   [junit4]   2> 3462088 INFO  (qtp1950205446-189793) [    x:00000_core] o.a.s.c.S.Request [00000_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=111
   [junit4]   2> 3462088 INFO  (qtp1950205446-189730) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id16]} 0 0
   [junit4]   2> 3462087 INFO  (qtp1950205446-189781) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id4]} 0 183
   [junit4]   2> 3462089 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id6]} 0 226
   [junit4]   2> 3462090 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id20]} 0 0
   [junit4]   2> 3462090 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462090 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@2f4a1627[00006_core] main]
   [junit4]   2> 3462090 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id21]} 0 0
   [junit4]   2> 3462091 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00006_core/conf
   [junit4]   2> 3462092 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462093 INFO  (searcherExecutor-15693-thread-1-processing-x:00006_core) [    x:00006_core] o.a.s.c.SolrCore [00006_core] Registered new searcher Searcher@2f4a1627[00006_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462093 INFO  (qtp1950205446-189778) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id5]} 0 191
   [junit4]   2> 3462102 INFO  (qtp1950205446-189792) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id13]} 0 40
   [junit4]   2> 3462102 INFO  (qtp1950205446-189784) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id14]} 0 40
   [junit4]   2> 3462102 INFO  (qtp1950205446-189723) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id22]} 0 0
   [junit4]   2> 3462103 INFO  (qtp1950205446-189789) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id12]} 0 43
   [junit4]   2> 3462103 INFO  (qtp1950205446-189773) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id24]} 0 0
   [junit4]   2> 3462103 INFO  (qtp1950205446-189775) [    x:00006_core] o.a.s.c.S.Request [00006_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=126
   [junit4]   2> 3462103 INFO  (qtp1950205446-189789) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id27]} 0 0
   [junit4]   2> 3462104 INFO  (qtp1950205446-189792) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id29]} 0 0
   [junit4]   2> 3462116 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3462116 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 3462140 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.s.IndexSchema [00001_core] Schema name=tiny
   [junit4]   2> 3462140 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.s.IndexSchema [00003_core] Schema name=tiny
   [junit4]   2> 3462143 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3462143 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 3462144 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.c.CoreContainer Creating SolrCore '00001_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00001_core, trusted=true
   [junit4]   2> 3462144 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.c.CoreContainer Creating SolrCore '00003_core' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00003_core, trusted=true
   [junit4]   2> 3462144 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00001_core' (registry 'solr.core.00001_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3462144 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.00003_core' (registry 'solr.core.00003_core') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67212aae
   [junit4]   2> 3462144 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.c.SolrCore [[00001_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00001_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00001_core/data/]
   [junit4]   2> 3462144 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.c.SolrCore [[00003_core] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00003_core], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00003_core/data/]
   [junit4]   2> 3462149 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@55ce3c2a
   [junit4]   2> 3462149 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7d9b1999
   [junit4]   2> 3462152 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462152 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462154 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462154 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@19db15d8[00002_core] main]
   [junit4]   2> 3462155 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00002_core/conf
   [junit4]   2> 3462156 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462161 INFO  (qtp1950205446-189789) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id28]} 0 56
   [junit4]   2> 3462161 INFO  (qtp1950205446-189775) [    x:00000_core] o.a.s.c.S.Request [00000_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 3462161 INFO  (qtp1950205446-189723) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id26]} 0 57
   [junit4]   2> 3462161 INFO  (qtp1950205446-189790) [    x:00004_core] o.a.s.c.S.Request [00004_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 3462161 INFO  (searcherExecutor-15696-thread-1-processing-x:00002_core) [    x:00002_core] o.a.s.c.SolrCore [00002_core] Registered new searcher Searcher@19db15d8[00002_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462162 INFO  (qtp1950205446-189790) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id31]} 0 0
   [junit4]   2> 3462162 INFO  (qtp1950205446-189773) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id32]} 0 0
   [junit4]   2> 3462162 INFO  (qtp1950205446-189784) [    x:00002_core] o.a.s.c.S.Request [00002_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=1
   [junit4]   2> 3462162 INFO  (qtp1950205446-189726) [    x:00002_core] o.a.s.c.S.Request [00002_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=197
   [junit4]   2> 3462162 INFO  (qtp1950205446-189790) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id33]} 0 0
   [junit4]   2> 3462162 INFO  (qtp1950205446-189729) [    x:00002_core] o.a.s.c.S.Request [00002_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=174
   [junit4]   2> 3462204 INFO  (qtp1950205446-189729) [    x:00000_core] o.a.s.c.S.Request [00000_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 3462229 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462229 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462230 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3462230 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3462230 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462231 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@59fdc0a5[00003_core] main]
   [junit4]   2> 3462231 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00003_core/conf
   [junit4]   2> 3462232 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=81.4150390625, floorSegmentMB=1.642578125, forceMergeDeletesPctAllowed=25.422157065380087, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6993008177502643, deletesPctAllowed=25.184345105296604
   [junit4]   2> 3462232 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@2054533[00001_core] main]
   [junit4]   2> 3462232 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462233 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_C4317875F2B7236A-001/tempDir-001/00001_core/conf
   [junit4]   2> 3462237 INFO  (searcherExecutor-15709-thread-1-processing-x:00003_core) [    x:00003_core] o.a.s.c.SolrCore [00003_core] Registered new searcher Searcher@59fdc0a5[00003_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462237 INFO  (qtp1950205446-189790) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id35]} 0 74
   [junit4]   2> 3462237 INFO  (qtp1950205446-189789) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id34]} 0 75
   [junit4]   2> 3462237 INFO  (qtp1950205446-189773) [    x:00003_core] o.a.s.c.S.Request [00003_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=48
   [junit4]   2> 3462237 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id15]} 0 149
   [junit4]   2> 3462238 INFO  (qtp1950205446-189783) [    x:00003_core] o.a.s.c.S.Request [00003_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=77
   [junit4]   2> 3462238 INFO  (qtp1950205446-189729) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id37]} 0 0
   [junit4]   2> 3462238 INFO  (qtp1950205446-189779) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id17]} 0 149
   [junit4]   2> 3462238 INFO  (qtp1950205446-189792) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id30]} 0 134
   [junit4]   2> 3462238 INFO  (qtp1950205446-189726) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id36]} 0 0
   [junit4]   2> 3462238 INFO  (qtp1950205446-189730) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id18]} 0 149
   [junit4]   2> 3462239 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3462239 INFO  (qtp1950205446-189791) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id19]} 0 149
   [junit4]   2> 3462239 INFO  (qtp1950205446-189783) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id41]} 0 0
   [junit4]   2> 3462239 INFO  (qtp1950205446-189779) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id40]} 0 0
   [junit4]   2> 3462240 INFO  (qtp1950205446-189783) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id43]} 0 0
   [junit4]   2> 3462240 INFO  (qtp1950205446-189789) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id42]} 0 0
   [junit4]   2> 3462240 INFO  (qtp1950205446-189779) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id44]} 0 0
   [junit4]   2> 3462240 INFO  (qtp1950205446-189790) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id45]} 0 0
   [junit4]   2> 3462240 INFO  (searcherExecutor-15708-thread-1-processing-x:00001_core) [    x:00001_core] o.a.s.c.SolrCore [00001_core] Registered new searcher Searcher@2054533[00001_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3462240 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id23]} 0 146
   [junit4]   2> 3462247 INFO  (qtp1950205446-189779) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id47]} 0 6
   [junit4]   2> 3462247 INFO  (qtp1950205446-189794) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id25]} 0 144
   [junit4]   2> 3462247 INFO  (qtp1950205446-189790) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id48]} 0 0
   [junit4]   2> 3462248 INFO  (qtp1950205446-189779) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id50]} 0 0
   [junit4]   2> 3462248 INFO  (qtp1950205446-189795) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id51]} 0 0
   [junit4]   2> 3462248 INFO  (qtp1950205446-189783) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id52]} 0 0
   [junit4]   2> 3462248 INFO  (qtp1950205446-189779) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id53]} 0 0
   [junit4]   2> 3462244 INFO  (qtp1950205446-189789) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id39]} 0 3
   [junit4]   2> 3462248 INFO  (qtp1950205446-189795) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id55]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189789) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id58]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189779) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id57]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189790) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id54]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189783) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id56]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189789) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id59]} 0 0
   [junit4]   2> 3462249 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id60]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189790) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id62]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189795) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id63]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189789) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id61]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189790) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id64]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189779) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id66]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189789) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id65]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189724) [    x:00001_core] o.a.s.c.S.Request [00001_core]  webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=89
   [junit4]   2> 3462250 INFO  (qtp1950205446-189727) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id69]} 0 0
   [junit4]   2> 3462250 INFO  (qtp1950205446-189795) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id67]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189789) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id49]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189795) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id70]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189784) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id46]} 0 7
   [junit4]   2> 3462251 INFO  (qtp1950205446-189724) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id73]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189795) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id68]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189783) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id71]} 0 0
   [junit4]   2> 3462251 INFO  (qtp1950205446-189789) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id75]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189779) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id72]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189727) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id74]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189784) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id76]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189780) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id38]} 0 14
   [junit4]   2> 3462252 INFO  (qtp1950205446-189724) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id79]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189789) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id80]} 0 0
   [junit4]   2> 3462252 INFO  (qtp1950205446-189727) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id77]} 0 0
   [junit4]   2> 3462253 INFO  (qtp1950205446-189780) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id83]} 0 0
   [junit4]   2> 3462253 INFO  (qtp1950205446-189724) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id84]} 0 0
   [junit4]   2> 3462253 INFO  (qtp1950205446-189789) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id86]} 0 0
   [junit4]   2> 3462253 INFO  (qtp1950205446-189780) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id87]} 0 0
   [junit4]   2> 3462253 INFO  (qtp1950205446-189795) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id81]} 0 1
   [junit4]   2> 3462253 INFO  (qtp1950205446-189789) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id89]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189795) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id88]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189780) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id90]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189789) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id91]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189780) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id93]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189795) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id94]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189789) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id95]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189795) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id92]} 0 0
   [junit4]   2> 3462254 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id96]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189789) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id97]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189780) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id99]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189795) [    x:00005_core] o.a.s.u.p.LogUpdateProcessorFactory [00005_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id100]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189784) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id98]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189789) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id102]} 0 0
   [junit4]   2> 3462255 INFO  (qtp1950205446-189724) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id85]} 0 2
   [junit4]   2> 3462255 INFO  (qtp1950205446-189780) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id103]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189724) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id105]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189780) [    x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id108]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189784) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id106]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189724) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id107]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189782) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id78]} 0 4
   [junit4]   2> 3462256 INFO  (qtp1950205446-189780) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id109]} 0 0
   [junit4]   2> 3462256 INFO  (qtp1950205446-189784) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id110]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189779) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id82]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189780) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id112]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189779) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id101]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id114]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189727) [    x:00001_core] o.a.s.u.p.LogUpdateProcessorFactory [00001_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id104]} 0 1
   [junit4]   2> 3462257 INFO  (qtp1950205446-189779) [    x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id113]} 0 0
   [junit4]   2> 3462257 INFO  (qtp1950205446-189780) [    x:00003_core] o.a.s.u.p.LogUpdateProcessorFactory [00003_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id119]} 0 0
   [junit4]   2> 3462258 INFO  (qtp1950205446-189784) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id111]} 0 1
   [junit4]   2> 3462258 INFO  (qtp1950205446-189782) [    x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[id118]} 0 0
   [junit4]   2> 3462258 INFO  (qtp1950205446-189782) [    x:00006_core] o.a.s.u.p.LogUpdateProcessorFactory [00006_core]  webapp=/solr path=/update params={wt=javabin&version=2}{a

[...truncated too long message...]

utor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=34646, name=searcherExecutor-11509-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    2) Thread[id=34681, name=searcherExecutor-11523-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    3) Thread[id=34716, name=searcherExecutor-11537-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    4) Thread[id=34495, name=searcherExecutor-11461-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    5) Thread[id=34576, name=searcherExecutor-11489-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    6) Thread[id=34532, name=searcherExecutor-11475-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    7) Thread[id=34453, name=searcherExecutor-11447-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C4317875F2B7236A]:0)
   [junit4] Completed [622/834 (2!)] on J0 in 455.65s, 1 test, 1 failure, 2 errors <<< FAILURES!

[...truncated 652 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20181020_140917_9556534085638260281975.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid25127.hprof ...
   [junit4] Heap dump file created [489741327 bytes in 1.422 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 8978 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid25127.hprof

Total time: 323 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any