You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/06/09 14:53:20 UTC
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 14 -
Failure
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/14/
9 tests failed.
FAILED: org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState
Error Message:
expected:<27> but was:<28>
Stack Trace:
java.lang.AssertionError: expected:<27> but was:<28>
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B:8D669BF4BA758976]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState(TestLeaderInitiatedRecoveryThread.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: junit.framework.TestSuite.org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
Error Message:
Captured an uncaught exception in thread: Thread[id=19234, name=Lucene Merge Thread #183, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19234, name=Lucene Merge Thread #183, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B]:0)
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: junit.framework.TestSuite.org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
Error Message:
Captured an uncaught exception in thread: Thread[id=19251, name=Lucene Merge Thread #200, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19251, name=Lucene Merge Thread #200, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B]:0)
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
Caused by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)
at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4974)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4502)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4052)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.newNode(HashMap.java:1747)
at java.util.HashMap.putVal(HashMap.java:631)
at java.util.HashMap.put(HashMap.java:612)
at java.util.HashSet.add(HashSet.java:220)
at org.apache.lucene.index.ReaderPool.noDups(ReaderPool.java:385)
at org.apache.lucene.index.ReaderPool.get(ReaderPool.java:365)
at org.apache.lucene.index.IndexWriter.getPooledInstance(IndexWriter.java:5174)
at org.apache.lucene.index.IndexWriter.numDeletedDocs(IndexWriter.java:640)
at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:4618)
at org.apache.lucene.index.IndexWriter$$Lambda$680/997265496.apply(Unknown Source)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:4610)
at org.apache.lucene.index.IndexWriter.handleMergeException(IndexWriter.java:4005)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4056)
... 2 more
FAILED: junit.framework.TestSuite.org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
Error Message:
Captured an uncaught exception in thread: Thread[id=19252, name=Lucene Merge Thread #201, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19252, name=Lucene Merge Thread #201, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B]:0)
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.<init>(String.java:207)
at java.lang.StringBuilder.toString(StringBuilder.java:407)
at java.util.StringJoiner.toString(StringJoiner.java:165)
at java.util.stream.Collectors$$Lambda$684/2046590577.apply(Unknown Source)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:503)
at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:4610)
at org.apache.lucene.index.IndexWriter.abortMerges(IndexWriter.java:2498)
at org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2278)
at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2265)
at org.apache.lucene.index.IndexWriter.maybeCloseOnTragicEvent(IndexWriter.java:4851)
at org.apache.lucene.index.IndexWriter.tragicEvent(IndexWriter.java:4841)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B:5B4F06D0B094AD73]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:921)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1478)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest: 1) Thread[id=29607, name=searcherExecutor-7494-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=29569, name=searcherExecutor-7480-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=29651, name=searcherExecutor-7508-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=29526, name=searcherExecutor-7466-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=29734, name=searcherExecutor-7528-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=29806, name=searcherExecutor-7556-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=29770, name=searcherExecutor-7542-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest:
1) Thread[id=29607, name=searcherExecutor-7494-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=29569, name=searcherExecutor-7480-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=29651, name=searcherExecutor-7508-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=29526, name=searcherExecutor-7466-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=29734, name=searcherExecutor-7528-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=29806, name=searcherExecutor-7556-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=29770, name=searcherExecutor-7542-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=29607, name=searcherExecutor-7494-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=29569, name=searcherExecutor-7480-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=29651, name=searcherExecutor-7508-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=29526, name=searcherExecutor-7466-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=29734, name=searcherExecutor-7528-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=29806, name=searcherExecutor-7556-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=29770, name=searcherExecutor-7542-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=29607, name=searcherExecutor-7494-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=29569, name=searcherExecutor-7480-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=29651, name=searcherExecutor-7508-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=29526, name=searcherExecutor-7466-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=29734, name=searcherExecutor-7528-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=29806, name=searcherExecutor-7556-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=29770, name=searcherExecutor-7542-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B]:0)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
Some docs had errors -- check logs expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: Some docs had errors -- check logs expected:<0> but was:<1>
at __randomizedtesting.SeedInfo.seed([D31B390A1E68C08B:E50F5B4C9435FA9A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:342)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:198)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13934 lines...]
[junit4] Suite: org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread
[junit4] 2> 2849977 INFO (SUITE-TestLeaderInitiatedRecoveryThread-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/init-core-data-001
[junit4] 2> 2849977 WARN (SUITE-TestLeaderInitiatedRecoveryThread-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 2849977 INFO (SUITE-TestLeaderInitiatedRecoveryThread-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2849985 INFO (SUITE-TestLeaderInitiatedRecoveryThread-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 2849985 INFO (SUITE-TestLeaderInitiatedRecoveryThread-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2849988 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2849989 INFO (Thread-11719) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2849989 INFO (Thread-11719) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2849998 ERROR (Thread-11719) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2850089 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.ZkTestServer start zk server on port:56927
[junit4] 2> 2850115 INFO (zkConnectionManagerCallback-3827-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850139 INFO (zkConnectionManagerCallback-3829-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850144 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2850145 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2850147 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2850148 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2850149 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2850151 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2850152 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2850153 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2850154 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2850155 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2850156 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2850158 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2850336 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cc2831f{/,null,AVAILABLE}
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6ef4d0c2{HTTP/1.1,[http/1.1]}{127.0.0.1:59711}
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.Server Started @2850587ms
[junit4] 2> 2850337 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=59711, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/control-001/cores}
[junit4] 2> 2850338 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2850338 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2850338 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2850338 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2850338 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2850338 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-09T13:46:53.127Z
[junit4] 2> 2850340 INFO (zkConnectionManagerCallback-3831-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850341 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2850341 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/control-001/solr.xml
[junit4] 2> 2850345 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2850345 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2850347 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2850547 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56927/solr
[junit4] 2> 2850549 INFO (zkConnectionManagerCallback-3835-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850551 INFO (zkConnectionManagerCallback-3837-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850687 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2850688 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59711_
[junit4] 2> 2850689 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.Overseer Overseer (id=73945287071891460-127.0.0.1:59711_-n_0000000000) starting
[junit4] 2> 2850723 INFO (zkConnectionManagerCallback-3844-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850725 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56927/solr ready
[junit4] 2> 2850746 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59711_
[junit4] 2> 2850758 INFO (zkCallback-3843-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2850767 INFO (zkCallback-3836-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2850800 WARN (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2850835 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2850858 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2850858 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2850860 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:59711_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/control-001/cores
[junit4] 2> 2850950 INFO (zkConnectionManagerCallback-3850-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2850951 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2850952 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56927/solr ready
[junit4] 2> 2850974 INFO (qtp240083452-26362) [n:127.0.0.1:59711_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:59711_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2850990 INFO (OverseerThreadFactory-6039-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2851303 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2851303 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2852344 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2852394 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2853012 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2853038 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2853038 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2853038 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2853038 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2853042 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3819e856
[junit4] 2> 2853062 WARN (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2853188 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2853188 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2853190 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2853190 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2853192 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=30.3291015625, floorSegmentMB=0.5244140625, forceMergeDeletesPctAllowed=11.722601373241035, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.7001953125, noCFSRatio=1.0
[junit4] 2> 2853200 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@fcf3b9b[control_collection_shard1_replica_n1] main]
[junit4] 2> 2853201 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2853202 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2853203 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2853204 INFO (searcherExecutor-6044-thread-1-processing-n:127.0.0.1:59711_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@fcf3b9b[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2853204 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1602802958721875968
[junit4] 2> 2853221 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59711/control_collection_shard1_replica_n1/
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:59711/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2853225 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2853229 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59711/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2853339 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2853348 INFO (qtp240083452-26357) [n:127.0.0.1:59711_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2045
[junit4] 2> 2853352 INFO (qtp240083452-26362) [n:127.0.0.1:59711_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2853448 INFO (zkCallback-3836-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2854352 INFO (qtp240083452-26362) [n:127.0.0.1:59711_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:59711_&wt=javabin&version=2} status=0 QTime=3377
[junit4] 2> 2854398 INFO (zkConnectionManagerCallback-3855-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2854399 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2854400 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56927/solr ready
[junit4] 2> 2854400 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2854401 INFO (qtp240083452-26358) [n:127.0.0.1:59711_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2854410 INFO (OverseerCollectionConfigSetProcessor-73945287071891460-127.0.0.1:59711_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2854411 INFO (OverseerThreadFactory-6039-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2854412 WARN (OverseerThreadFactory-6039-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2854620 INFO (qtp240083452-26358) [n:127.0.0.1:59711_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2854620 INFO (qtp240083452-26358) [n:127.0.0.1:59711_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=219
[junit4] 2> 2854813 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001 of type NRT
[junit4] 2> 2854814 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2854815 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2854815 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2854815 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2854816 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@207b61af{/,null,AVAILABLE}
[junit4] 2> 2854816 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@76566f11{HTTP/1.1,[http/1.1]}{127.0.0.1:51563}
[junit4] 2> 2854816 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.e.j.s.Server Started @2855066ms
[junit4] 2> 2854816 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=51563, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001/cores}
[junit4] 2> 2854817 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2854817 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2854817 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2854817 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2854817 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2854817 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-09T13:46:57.606Z
[junit4] 2> 2854819 INFO (zkConnectionManagerCallback-3857-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2854820 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2854820 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001/solr.xml
[junit4] 2> 2854824 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2854824 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2854826 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2855016 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56927/solr
[junit4] 2> 2855030 INFO (zkConnectionManagerCallback-3861-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2855210 INFO (zkConnectionManagerCallback-3863-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2855216 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2855218 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2855232 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2855232 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51563_
[junit4] 2> 2855234 INFO (zkCallback-3843-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2855234 INFO (zkCallback-3836-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2855262 INFO (zkCallback-3862-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2855274 INFO (zkCallback-3854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2855326 INFO (zkConnectionManagerCallback-3870-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2855327 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2855328 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56927/solr ready
[junit4] 2> 2855334 WARN (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2855378 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2855409 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2855410 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2855422 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [n:127.0.0.1:51563_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001/cores
[junit4] 2> 2855472 INFO (qtp2067180293-26418) [n:127.0.0.1:51563_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:51563_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2855475 INFO (OverseerCollectionConfigSetProcessor-73945287071891460-127.0.0.1:59711_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2855476 INFO (OverseerThreadFactory-6039-thread-3) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:51563_ for creating new replica
[junit4] 2> 2855480 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2856499 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2856564 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 2856818 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2856836 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 2856836 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10bf574a
[junit4] 2> 2856836 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2856836 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 2856847 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2987dae5
[junit4] 2> 2856883 WARN (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2857059 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2857059 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2857061 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2857061 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2857062 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=30.3291015625, floorSegmentMB=0.5244140625, forceMergeDeletesPctAllowed=11.722601373241035, segmentsPerTier=31.0, maxCFSSegmentSizeMB=1.7001953125, noCFSRatio=1.0
[junit4] 2> 2857068 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@57e344d0[collection1_shard1_replica_n21] main]
[junit4] 2> 2857069 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2857069 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2857070 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2857071 INFO (searcherExecutor-6058-thread-1-processing-n:127.0.0.1:51563_ x:collection1_shard1_replica_n21 c:collection1 s:shard1) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@57e344d0[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2857071 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1602802962776719360
[junit4] 2> 2857205 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51563/collection1_shard1_replica_n21/
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy http://127.0.0.1:51563/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 2857208 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2857211 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51563/collection1_shard1_replica_n21/ shard1
[junit4] 2> 2857362 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2857365 INFO (qtp2067180293-26413) [n:127.0.0.1:51563_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1885
[junit4] 2> 2857369 INFO (qtp2067180293-26418) [n:127.0.0.1:51563_ c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:51563_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1896
[junit4] 2> 2857479 INFO (OverseerCollectionConfigSetProcessor-73945287071891460-127.0.0.1:59711_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2857535 INFO (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[D31B390A1E68C08B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_D31B390A1E68C08B-001/shard-2-001 of type NRT
[junit4] 2
[...truncated too long message...]
eUpdate-73945446353141772-127.0.0.1:40427_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40427_solr
[junit4] 2> 5575203 WARN (OverseerAutoScalingTriggerThread-73945446353141772-127.0.0.1:40427_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 5575243 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x106b4fddea30018, likely client has closed socket
[junit4] 2> 5575243 INFO (jetty-closer-4996-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4b8dc97a{/solr,null,UNAVAILABLE}
[junit4] 2> 5575247 INFO (jetty-closer-4996-thread-4) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5575247 INFO (jetty-closer-4996-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6d41b114{/solr,null,UNAVAILABLE}
[junit4] 2> 5575264 INFO (zkCallback-5046-thread-3) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37549_solr
[junit4] 2> 5575265 INFO (jetty-closer-4996-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7c7911b1{/solr,null,UNAVAILABLE}
[junit4] 2> 5575266 INFO (jetty-closer-4996-thread-5) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5575266 INFO (jetty-closer-4996-thread-9) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3343ae31{/solr,null,UNAVAILABLE}
[junit4] 2> 5575272 INFO (jetty-closer-4996-thread-9) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5575272 INFO (jetty-closer-4996-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5575282 INFO (jetty-closer-4996-thread-10) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2bd012de{/solr,null,UNAVAILABLE}
[junit4] 2> 5575283 INFO (jetty-closer-4996-thread-10) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5575345 INFO (jetty-closer-4996-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@767e768a{/solr,null,UNAVAILABLE}
[junit4] 2> 5575346 INFO (zkCallback-5148-thread-4) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36015_solr
[junit4] 2> 5575348 INFO (jetty-closer-4996-thread-3) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5576847 INFO (jetty-closer-4996-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@773ee4df{/solr,null,UNAVAILABLE}
[junit4] 2> 5576848 INFO (jetty-closer-4996-thread-8) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5576851 ERROR (SUITE-TestStressCloudBlindAtomicUpdates-seed#[D31B390A1E68C08B]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 5576851 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[D31B390A1E68C08B]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39089 39089
[junit4] 2> 5577030 INFO (Thread-11357) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39089 39089
[junit4] 2> 5577031 WARN (Thread-11357) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 101 /solr/collections/test_col/terms/shard3
[junit4] 2> 93 /solr/collections/test_col/terms/shard5
[junit4] 2> 89 /solr/collections/test_col/terms/shard1
[junit4] 2> 87 /solr/collections/test_col/terms/shard4
[junit4] 2> 79 /solr/collections/test_col/terms/shard2
[junit4] 2> 21 /solr/aliases.json
[junit4] 2> 21 /solr/clusterprops.json
[junit4] 2> 10 /solr/configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
[junit4] 2> 10 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 21 /solr/clusterstate.json
[junit4] 2> 10 /solr/collections/test_col/state.json
[junit4] 2> 4 /solr/overseer_elect/election/73945446353141777-127.0.0.1:37549_solr-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73945446353141791-127.0.0.1:55089_solr-n_0000000006
[junit4] 2> 2 /solr/overseer_elect/election/73945446353141781-127.0.0.1:55829_solr-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73945446353141789-127.0.0.1:46547_solr-n_0000000005
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 21 /solr/live_nodes
[junit4] 2> 21 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_D31B390A1E68C08B-001
[junit4] 2> Jun 09, 2018 2:32:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {long_stored_idx=PostingsFormat(name=LuceneVarGapFixedInterval), long_dv_idx=FSTOrd50, id=FSTOrd50, long_dv_stored_idx=FSTOrd50}, docValues:{_version_=DocValuesFormat(name=Direct), long_dv_idx=DocValuesFormat(name=Lucene70), long_dv_stored=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), long_dv=DocValuesFormat(name=Lucene70), long_dv_stored_idx=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=619, maxMBSortInHeap=5.570579582217124, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7ca0098c), locale=id, timezone=Europe/Helsinki
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=119503344,total=514850816
[junit4] 2> NOTE: All tests run in this JVM: [SearchRateTriggerTest, TestUtilizeNode, TermVectorComponentDistributedTest, TestUseDocValuesAsStored, TestHashQParserPlugin, TestFieldResource, EchoParamsTest, SuggesterWFSTTest, TestPhraseSuggestions, TestEmbeddedSolrServerAdminHandler, RegexBytesRefFilterTest, TestIndexSearcher, BlockJoinFacetSimpleTest, TestMacroExpander, TestQuerySenderNoQuery, SolrCloudReportersTest, SuggestComponentTest, AddReplicaTest, TestLuceneMatchVersion, TestPayloadScoreQParserPlugin, OutputWriterTest, SharedFSAutoReplicaFailoverTest, TriggerCooldownIntegrationTest, DefaultValueUpdateProcessorTest, TestRequestStatusCollectionAPI, TestDynamicFieldCollectionResource, ConfigureRecoveryStrategyTest, TestCloudDeleteByQuery, RollingRestartTest, SolrRequestParserTest, TestFieldCacheSort, ClassificationUpdateProcessorIntegrationTest, SmileWriterTest, AtomicUpdateProcessorFactoryTest, HLLSerializationTest, TestHighlightDedupGrouping, TestJsonRequest, SolrCoreMetricManagerTest, TestFieldCacheWithThreads, SolrPluginUtilsTest, EnumFieldTest, PKIAuthenticationIntegrationTest, TestCollectionAPIs, TestSchemaManager, HttpSolrCallGetCoreTest, SolrIndexConfigTest, TestReRankQParserPlugin, TestNestedDocsSort, TestJoin, TestLockTree, BinaryUpdateRequestHandlerTest, ScheduledTriggerIntegrationTest, TestHttpShardHandlerFactory, TestRestoreCore, TestSort, TestExecutePlanAction, TestImpersonationWithHadoopAuth, TestManagedSchemaAPI, TestDynamicURP, UpdateParamsTest, ProtectedTermFilterFactoryTest, OpenCloseCoreStressTest, OverseerCollectionConfigSetProcessorTest, TestClusterStateProvider, TestCryptoKeys, TestNumericTerms64, TestSolrXml, DistribDocExpirationUpdateProcessorTest, TestRandomFaceting, CursorPagingTest, TestClusterStateMutator, NodeMutatorTest, CheckHdfsIndexTest, ChaosMonkeyNothingIsSafeTest, DeleteShardTest, TestShardHandlerFactory, DistributedFacetPivotSmallAdvancedTest, TestChildDocTransformer, TestCSVResponseWriter, ShardRoutingTest, HdfsDirectoryTest, RequestHandlersTest, DistributedFacetExistsSmallTest, HighlighterConfigTest, BJQParserTest, TestHdfsCloudBackupRestore, TestWordDelimiterFilterFactory, TestDocTermOrdsUninvertLimit, TestWriterPerf, TestCollationField, SchemaApiFailureTest, BasicDistributedZkTest, TestTlogReplica, HdfsUnloadDistributedZkTest, NumberUtilsTest, ReplaceNodeTest, TestDistributedMap, CreateCollectionCleanupTest, DistributedVersionInfoTest, TestRuleBasedAuthorizationPlugin, SortSpecParsingTest, TestIntervalFaceting, SolrXmlInZkTest, HDFSCollectionsAPITest, CustomHighlightComponentTest, TestStressCloudBlindAtomicUpdates]
[junit4] Completed [791/818 (4!)] on J0 in 296.90s, 5 tests, 1 failure <<< FAILURES!
[...truncated 79 lines...]
[junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180609_125922_4071606005963904416862.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/heapdumps/java_pid23900.hprof ...
[junit4] Heap dump file created [470051636 bytes in 51.366 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 9410 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid23900.hprof
Total time: 384 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 25 -
Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/25/
3 tests failed.
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([16787B48039FC63D:9E2C4492AD63ABC5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:921)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1478)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest: 1) Thread[id=34455, name=searcherExecutor-10489-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=34411, name=searcherExecutor-10475-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=34632, name=searcherExecutor-10537-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=34668, name=searcherExecutor-10551-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=34707, name=searcherExecutor-10565-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=34493, name=searcherExecutor-10503-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=34537, name=searcherExecutor-10517-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest:
1) Thread[id=34455, name=searcherExecutor-10489-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=34411, name=searcherExecutor-10475-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=34632, name=searcherExecutor-10537-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=34668, name=searcherExecutor-10551-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=34707, name=searcherExecutor-10565-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=34493, name=searcherExecutor-10503-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=34537, name=searcherExecutor-10517-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([16787B48039FC63D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=34455, name=searcherExecutor-10489-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=34411, name=searcherExecutor-10475-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=34632, name=searcherExecutor-10537-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=34668, name=searcherExecutor-10551-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=34707, name=searcherExecutor-10565-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=34493, name=searcherExecutor-10503-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=34537, name=searcherExecutor-10517-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=34455, name=searcherExecutor-10489-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=34411, name=searcherExecutor-10475-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=34632, name=searcherExecutor-10537-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=34668, name=searcherExecutor-10551-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=34707, name=searcherExecutor-10565-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=34493, name=searcherExecutor-10503-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=34537, name=searcherExecutor-10517-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([16787B48039FC63D]:0)
Build Log:
[...truncated 14406 lines...]
[junit4] Suite: org.apache.solr.cloud.RestartWhileUpdatingTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/init-core-data-001
[junit4] 2> 3331576 WARN (SUITE-RestartWhileUpdatingTest-seed#[16787B48039FC63D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
[junit4] 2> 3331576 INFO (SUITE-RestartWhileUpdatingTest-seed#[16787B48039FC63D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3331578 INFO (SUITE-RestartWhileUpdatingTest-seed#[16787B48039FC63D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3331578 INFO (SUITE-RestartWhileUpdatingTest-seed#[16787B48039FC63D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3331578 INFO (SUITE-RestartWhileUpdatingTest-seed#[16787B48039FC63D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_/so
[junit4] 2> 3331580 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3331581 INFO (Thread-10369) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3331581 INFO (Thread-10369) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3331583 ERROR (Thread-10369) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3331681 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkTestServer start zk server on port:41239
[junit4] 2> 3331684 INFO (zkConnectionManagerCallback-7997-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3331688 INFO (zkConnectionManagerCallback-7999-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3331692 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3331694 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 3331695 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3331697 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3331698 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3331700 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3331702 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3331703 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3331705 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3331707 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3331709 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3331711 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 3331840 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 3331840 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3331840 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3331840 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 3331842 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10c23f85{/_/so,null,AVAILABLE}
[junit4] 2> 3331844 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@354f5efd{HTTP/1.1,[http/1.1]}{127.0.0.1:45217}
[junit4] 2> 3331844 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server Started @3331892ms
[junit4] 2> 3331844 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/tempDir-001/control/data, hostContext=/_/so, hostPort=45217, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/control-001/cores}
[junit4] 2> 3331845 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3331845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3331845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 3331845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3331845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3331845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-24T17:15:59.810Z
[junit4] 2> 3331847 INFO (zkConnectionManagerCallback-8001-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3331848 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3331848 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/control-001/solr.xml
[junit4] 2> 3331852 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 3331852 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 3331853 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 3332037 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41239/solr
[junit4] 2> 3332038 INFO (zkConnectionManagerCallback-8005-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3332040 INFO (zkConnectionManagerCallback-8007-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3332156 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45217__%2Fso
[junit4] 2> 3332156 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.Overseer Overseer (id=72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) starting
[junit4] 2> 3332165 INFO (zkConnectionManagerCallback-8014-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3332168 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41239/solr ready
[junit4] 2> 3332169 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45217__%2Fso
[junit4] 2> 3332169 INFO (OverseerStateUpdate-72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:45217__%2Fso
[junit4] 2> 3332178 INFO (OverseerStateUpdate-72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3332180 INFO (zkCallback-8013-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3332197 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3332217 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3332228 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3332228 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3332229 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/control-001/cores
[junit4] 2> 3332252 INFO (zkConnectionManagerCallback-8020-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3332253 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3332254 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41239/solr ready
[junit4] 2> 3332256 INFO (qtp1813844423-34370) [n:127.0.0.1:45217__%2Fso ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45217__%252Fso&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3332260 INFO (OverseerThreadFactory-10470-thread-1-processing-n:127.0.0.1:45217__%2Fso) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 3332368 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3332368 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3333386 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3333399 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 3333478 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3333493 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 3333493 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3333494 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3333494 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 3333497 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5ce130c2
[junit4] 2> 3333504 WARN (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3333557 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3333557 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3333558 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 3333558 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 3333560 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@59b3cdc1
[junit4] 2> 3333561 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@58e66ea8[control_collection_shard1_replica_n1] main]
[junit4] 2> 3333562 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3333562 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3333563 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3333563 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1609701483115184128
[junit4] 2> 3333564 INFO (searcherExecutor-10475-thread-1-processing-n:127.0.0.1:45217__%2Fso x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@58e66ea8[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3333575 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 3333578 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3333579 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3333579 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45217/_/so/control_collection_shard1_replica_n1/
[junit4] 2> 3333579 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3333579 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:45217/_/so/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 3333579 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3333582 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45217/_/so/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 3333685 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3333688 INFO (qtp1813844423-34365) [n:127.0.0.1:45217__%2Fso c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1320
[junit4] 2> 3333695 INFO (qtp1813844423-34370) [n:127.0.0.1:45217__%2Fso ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3333789 INFO (zkCallback-8006-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3334266 INFO (OverseerCollectionConfigSetProcessor-72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3334695 INFO (qtp1813844423-34370) [n:127.0.0.1:45217__%2Fso ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45217__%252Fso&wt=javabin&version=2} status=0 QTime=2439
[junit4] 2> 3334703 INFO (zkConnectionManagerCallback-8025-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3334705 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3334706 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41239/solr ready
[junit4] 2> 3334706 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3334707 INFO (qtp1813844423-34372) [n:127.0.0.1:45217__%2Fso ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3334710 INFO (OverseerThreadFactory-10470-thread-2-processing-n:127.0.0.1:45217__%2Fso) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 3334917 WARN (OverseerThreadFactory-10470-thread-2-processing-n:127.0.0.1:45217__%2Fso) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 3334922 INFO (qtp1813844423-34372) [n:127.0.0.1:45217__%2Fso ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3334922 INFO (qtp1813844423-34372) [n:127.0.0.1:45217__%2Fso ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=214
[junit4] 2> 3335063 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001 of type NRT
[junit4] 2> 3335064 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 3335065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3335065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3335065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 3335065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@247842bf{/_/so,null,AVAILABLE}
[junit4] 2> 3335065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1fa22fb3{HTTP/1.1,[http/1.1]}{127.0.0.1:45075}
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server Started @3335113ms
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/_/so, hostPort=45075, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001/cores}
[junit4] 2> 3335066 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3335066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-24T17:16:03.031Z
[junit4] 2> 3335068 INFO (zkConnectionManagerCallback-8027-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3335069 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3335069 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001/solr.xml
[junit4] 2> 3335073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 3335073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 3335075 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 3335190 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41239/solr
[junit4] 2> 3335192 INFO (zkConnectionManagerCallback-8031-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3335194 INFO (zkConnectionManagerCallback-8033-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3335201 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3335205 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3335205 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45075__%2Fso
[junit4] 2> 3335207 INFO (zkCallback-8013-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3335207 INFO (zkCallback-8024-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3335207 INFO (zkCallback-8006-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3335207 INFO (zkCallback-8032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3335232 INFO (zkConnectionManagerCallback-8040-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3335233 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3335234 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41239/solr ready
[junit4] 2> 3335235 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3335271 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3335282 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3335283 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3335284 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001/cores
[junit4] 2> 3335316 INFO (qtp2100306306-34426) [n:127.0.0.1:45075__%2Fso ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45075__%252Fso&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3335319 INFO (OverseerCollectionConfigSetProcessor-72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3335319 INFO (OverseerThreadFactory-10470-thread-3-processing-n:127.0.0.1:45217__%2Fso) [n:127.0.0.1:45217__%2Fso c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45075__%2Fso for creating new replica of shard shard1
[junit4] 2> 3335322 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3336341 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3336354 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 3336438 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3336455 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 3336455 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3336455 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3336455 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 3336459 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@367eef0e
[junit4] 2> 3336467 WARN (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3336515 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3336515 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3336517 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 3336517 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 3336519 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7a9c904e
[junit4] 2> 3336520 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@afdbb22[collection1_shard1_replica_n21] main]
[junit4] 2> 3336521 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3336521 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3336523 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3336523 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1609701486218969088
[junit4] 2> 3336524 INFO (searcherExecutor-10489-thread-1-processing-n:127.0.0.1:45075__%2Fso x:collection1_shard1_replica_n21 c:collection1 s:shard1) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@afdbb22[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3336530 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45075/_/so/collection1_shard1_replica_n21/
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy http://127.0.0.1:45075/_/so/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 3336533 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3336536 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45075/_/so/collection1_shard1_replica_n21/ shard1
[junit4] 2> 3336688 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3336690 INFO (qtp2100306306-34421) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1368
[junit4] 2> 3336692 INFO (qtp2100306306-34426) [n:127.0.0.1:45075__%2Fso c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45075__%252Fso&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1376
[junit4] 2> 3336816 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001 of type NRT
[junit4] 2> 3336816 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 3336817 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3336817 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3336817 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 3336818 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36036055{/_/so,null,AVAILABLE}
[junit4] 2> 3336818 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@10c32365{HTTP/1.1,[http/1.1]}{127.0.0.1:38874}
[junit4] 2> 3336818 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.e.j.s.Server Started @3336866ms
[junit4] 2> 3336818 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/_/so, hostPort=38874, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001/cores}
[junit4] 2> 3336818 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3336819 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3336819 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 3336819 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3336819 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3336819 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-24T17:16:04.784Z
[junit4] 2> 3336821 INFO (zkConnectionManagerCallback-8043-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3336822 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3336822 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001/solr.xml
[junit4] 2> 3336826 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 3336826 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 3336828 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 3337013 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41239/solr
[junit4] 2> 3337014 INFO (zkConnectionManagerCallback-8047-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3337017 INFO (zkConnectionManagerCallback-8049-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3337023 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3337028 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3337028 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38874__%2Fso
[junit4] 2> 3337029 INFO (zkCallback-8024-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337030 INFO (zkCallback-8039-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337030 INFO (zkCallback-8032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337030 INFO (zkCallback-8013-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337030 INFO (zkCallback-8006-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337030 INFO (zkCallback-8048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3337052 INFO (zkConnectionManagerCallback-8056-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3337053 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3337054 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41239/solr ready
[junit4] 2> 3337054 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3337077 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3337088 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3337088 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3337090 INFO (TEST-RestartWhileUpdatingTest.test-seed#[16787B48039FC63D]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001/cores
[junit4] 2> 3337142 INFO (qtp591077909-34464) [n:127.0.0.1:38874__%2Fso ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:38874__%252Fso&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3337144 INFO (OverseerCollectionConfigSetProcessor-72216172901236740-127.0.0.1:45217__%2Fso-n_0000000000) [n:127.0.0.1:45217__%2Fso ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3337144 INFO (OverseerThreadFactory-10470-thread-4-processing-n:127.0.0.1:45217__%2Fso) [n:127.0.0.1:45217__%2Fso c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:38874__%2Fso for creating new replica of shard shard1
[junit4] 2> 3337147 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3338162 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3338174 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 3338254 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3338274 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, trusted=true
[junit4] 2> 3338274 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n23' (registry 'solr.core.collection1.shard1.replica_n23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@439ea36e
[junit4] 2> 3338274 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3338274 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [[collection1_shard1_replica_n23] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001/cores/collection1_shard1_replica_n23], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RestartWhileUpdatingTest_16787B48039FC63D-001/shard-2-001/cores/collection1_shard1_replica_n23/data/]
[junit4] 2> 3338278 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@42746fa5
[junit4] 2> 3338285 WARN (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3338332 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3338332 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3338333 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 3338333 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 3338335 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@47cbb91e
[junit4] 2> 3338335 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a35e24d[collection1_shard1_replica_n23] main]
[junit4] 2> 3338336 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3338337 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3338337 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3338337 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1609701488121085952
[junit4] 2> 3338338 INFO (searcherExecutor-10503-thread-1-processing-n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23 c:collection1 s:shard1) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [collection1_shard1_replica_n23] Registered new searcher Searcher@4a35e24d[collection1_shard1_replica_n23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3338342 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node24=0, core_node22=0}, version=1}
[junit4] 2> 3338343 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n23
[junit4] 2> 3338344 INFO (updateExecutor-8044-thread-1-processing-n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23 c:collection1 s:shard1) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 3338344 INFO (recoveryExecutor-8045-thread-1-processing-n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23 c:collection1 s:shard1 r:core_node24) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 3338344 INFO (recoveryExecutor-8045-thread-1-processing-n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23 c:collection1 s:shard1 r:core_node24) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 3338345 INFO (qtp591077909-34459) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1198
[junit4] 2> 3338346 INFO (qtp2100306306-34422) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.c.S.Request [collection1_shard1_replica_n21] webapp=/_/so path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 3338347 INFO (qtp2100306306-34422) [n:127.0.0.1:45075__%2Fso c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.c.S.Request [collection1_shard1_replica_n21] webapp=/_/so path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3338347 INFO (recoveryExecutor-8045-thread-1-processing-n:127.0.0.1:38874__%2Fso x:collection1_shard1_replica_n23 c:collection1 s:shard1 r:core_node24) [n:127.0.0.1:38874__%2Fso c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_
[...truncated too long message...]
figure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null1417660032
[copy] Copying 238 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null1417660032
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null1417660032
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 70ms :: artifacts dl 3ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 313 minutes 41 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 24 -
Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/24/
9 tests failed.
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:7FFB6421A6578755]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:1C95AF511D3F284]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries(CdcrReplicationDistributedZkTest.java:567)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:54FCFA04C06E324F]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete(CdcrReplicationDistributedZkTest.java:534)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:F116CF4336D89551]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange(CdcrReplicationDistributedZkTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:8384113AB82EF626]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget(CdcrReplicationDistributedZkTest.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:A0255D337CAFB8EA]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:DD89DB03AA571072]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation(CdcrReplicationDistributedZkTest.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:3F9E1F43274438BE]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps(CdcrReplicationDistributedZkTest.java:471)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([23E683A068773363:23AC8C973E474BE9]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader(CdcrReplicationDistributedZkTest.java:441)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14145 lines...]
[junit4] Suite: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest
[junit4] 2> 2888746 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[23E683A068773363]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/init-core-data-001
[junit4] 2> 2888746 WARN (SUITE-CdcrReplicationDistributedZkTest-seed#[23E683A068773363]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 2888747 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[23E683A068773363]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2888748 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[23E683A068773363]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2888749 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[23E683A068773363]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2888752 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2888752 INFO (Thread-118551) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2888752 INFO (Thread-118551) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2888754 ERROR (Thread-118551) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2888852 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkTestServer start zk server on port:37636
[junit4] 2> 2888856 INFO (zkConnectionManagerCallback-1308-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2888859 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1006bd3db400000, likely client has closed socket
[junit4] 2> 2888869 INFO (zkConnectionManagerCallback-1310-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2888876 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2888878 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2888881 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2888883 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2888884 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2888886 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2888887 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2888889 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2888891 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2888894 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2888896 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2888900 INFO (zkConnectionManagerCallback-1314-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2888903 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplicationAfterRestart
[junit4] 2> 2889092 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 2889094 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2889094 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2889094 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2889097 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33fa7af4{/,null,AVAILABLE}
[junit4] 2> 2889106 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2564a94{SSL,[ssl, http/1.1]}{127.0.0.1:42516}
[junit4] 2> 2889106 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server Started @2889158ms
[junit4] 2> 2889106 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/, hostPort=42516, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-001/cores}
[junit4] 2> 2889106 ERROR (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2889107 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2889107 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2889107 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2889107 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2889107 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-17T15:38:03.214Z
[junit4] 2> 2889113 INFO (zkConnectionManagerCallback-1316-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889114 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2889114 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-001/solr.xml
[junit4] 2> 2889123 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2889123 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2889126 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2889222 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37636/solr
[junit4] 2> 2889224 INFO (zkConnectionManagerCallback-1320-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889227 INFO (zkConnectionManagerCallback-1322-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889315 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42516_
[junit4] 2> 2889316 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.c.Overseer Overseer (id=72176151698604037-127.0.0.1:42516_-n_0000000000) starting
[junit4] 2> 2889322 INFO (zkConnectionManagerCallback-1329-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889324 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37636/solr ready
[junit4] 2> 2889325 INFO (OverseerStateUpdate-72176151698604037-127.0.0.1:42516_-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42516_
[junit4] 2> 2889326 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42516_
[junit4] 2> 2889337 INFO (zkCallback-1321-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2889338 INFO (zkCallback-1328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2889351 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2889371 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889383 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889384 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889386 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [n:127.0.0.1:42516_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-001/cores
[junit4] 2> 2889551 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 2889553 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2889553 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2889553 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2889553 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@581f5493{/,null,AVAILABLE}
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1aa79895{SSL,[ssl, http/1.1]}{127.0.0.1:43000}
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server Started @2889606ms
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/, hostPort=43000, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-002/cores}
[junit4] 2> 2889554 ERROR (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2889554 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-17T15:38:03.661Z
[junit4] 2> 2889556 INFO (zkConnectionManagerCallback-1332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889558 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2889558 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-002/solr.xml
[junit4] 2> 2889569 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2889569 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2889577 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2889756 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37636/solr
[junit4] 2> 2889758 INFO (zkConnectionManagerCallback-1336-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889760 INFO (zkConnectionManagerCallback-1338-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889766 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2889770 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2889770 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43000_
[junit4] 2> 2889777 INFO (zkCallback-1321-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2889777 INFO (zkCallback-1328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2889777 INFO (zkCallback-1337-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2889804 INFO (zkConnectionManagerCallback-1345-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889805 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2889806 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37636/solr ready
[junit4] 2> 2889807 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2889825 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889835 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889835 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2889837 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-002/cores
[junit4] 2> 2889985 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 2889986 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2889986 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2889986 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2889987 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77772207{/,null,AVAILABLE}
[junit4] 2> 2889988 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@39831246{SSL,[ssl, http/1.1]}{127.0.0.1:45934}
[junit4] 2> 2889988 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server Started @2890040ms
[junit4] 2> 2889988 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard3, hostContext=/, hostPort=45934, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-003/cores}
[junit4] 2> 2889988 ERROR (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2889988 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2889988 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2889989 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2889989 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2889989 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-17T15:38:04.096Z
[junit4] 2> 2889991 INFO (zkConnectionManagerCallback-1348-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2889992 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2889992 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-003/solr.xml
[junit4] 2> 2889997 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2889997 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2889999 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2890130 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37636/solr
[junit4] 2> 2890131 INFO (zkConnectionManagerCallback-1352-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890133 INFO (zkConnectionManagerCallback-1354-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890139 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2890143 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2890143 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45934_
[junit4] 2> 2890144 INFO (zkCallback-1328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2890144 INFO (zkCallback-1337-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2890144 INFO (zkCallback-1321-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2890145 INFO (zkCallback-1344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2890145 INFO (zkCallback-1353-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2890165 INFO (zkConnectionManagerCallback-1361-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890166 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2890167 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37636/solr ready
[junit4] 2> 2890167 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2890186 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890195 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890196 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890197 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-003/cores
[junit4] 2> 2890345 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 2890346 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2890346 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2890346 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2890347 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f52fff7{/,null,AVAILABLE}
[junit4] 2> 2890347 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e7ad182{SSL,[ssl, http/1.1]}{127.0.0.1:41958}
[junit4] 2> 2890347 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.e.j.s.Server Started @2890399ms
[junit4] 2> 2890347 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard4, hostContext=/, hostPort=41958, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-004/cores}
[junit4] 2> 2890348 ERROR (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2890348 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2890348 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2890348 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2890348 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2890349 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-17T15:38:04.455Z
[junit4] 2> 2890352 INFO (zkConnectionManagerCallback-1364-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890354 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2890354 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-004/solr.xml
[junit4] 2> 2890360 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2890360 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2890361 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2890445 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37636/solr
[junit4] 2> 2890447 INFO (zkConnectionManagerCallback-1368-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890449 INFO (zkConnectionManagerCallback-1370-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890455 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2890459 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2890459 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41958_
[junit4] 2> 2890460 INFO (zkCallback-1328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890460 INFO (zkCallback-1337-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890460 INFO (zkCallback-1353-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890461 INFO (zkCallback-1344-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890461 INFO (zkCallback-1321-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890461 INFO (zkCallback-1360-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890461 INFO (zkCallback-1369-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2890480 INFO (zkConnectionManagerCallback-1377-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890481 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2890482 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37636/solr ready
[junit4] 2> 2890482 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2890503 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890512 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890513 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2890514 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-004/cores
[junit4] 2> 2890557 INFO (zkConnectionManagerCallback-1383-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2890558 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2890559 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[23E683A068773363]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37636/solr ready
[junit4] 2> 2890573 INFO (qtp1092877273-123710) [n:127.0.0.1:42516_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=tmp_collection&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2890576 INFO (OverseerThreadFactory-2005-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection tmp_collection
[junit4] 2> 2890684 WARN (OverseerThreadFactory-2005-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (tmp_collection) without cores.
[junit4] 2> 2890687 INFO (qtp1092877273-123710) [n:127.0.0.1:42516_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2890687 INFO (qtp1092877273-123710) [n:127.0.0.1:42516_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=tmp_collection&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&wt=javabin&version=2} status=0 QTime=113
[junit4] 2> 2890688 INFO (qtp1092877273-123709) [n:127.0.0.1:42516_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42516_&action=ADDREPLICA&collection=tmp_collection&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2890691 INFO (OverseerCollectionConfigSetProcessor-72176151698604037-127.0.0.1:42516_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2890692 INFO (OverseerThreadFactory-2005-thread-2) [ c:tmp_collection s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42516_ for creating new replica of shard shard1
[junit4] 2> 2890697 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ x:tmp_collection_shard1_replica_n41] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=tmp_collection_shard1_replica_n41&action=CREATE&collection=tmp_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2890698 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ x:tmp_collection_shard1_replica_n41] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2891727 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2891738 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.s.IndexSchema [tmp_collection_shard1_replica_n41] Schema name=test
[junit4] 2> 2891805 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2891820 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'tmp_collection_shard1_replica_n41' using configuration from collection tmp_collection, trusted=true
[junit4] 2> 2891820 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.tmp_collection.shard1.replica_n41' (registry 'solr.core.tmp_collection.shard1.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5bbf96c
[junit4] 2> 2891820 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2891820 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.c.SolrCore [[tmp_collection_shard1_replica_n41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-001/cores/tmp_collection_shard1_replica_n41], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_23E683A068773363-001/jetty-001/cores/tmp_collection_shard1_replica_n41/data/]
[junit4] 2> 2891823 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2891920 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 2891921 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2891922 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2891922 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2891923 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=37.8759765625, floorSegmentMB=0.9833984375, forceMergeDeletesPctAllowed=13.752133915027109, segmentsPerTier=25.0, maxCFSSegmentSizeMB=0.5712890625, noCFSRatio=0.0, deletesPctAllowed=43.63232293985087
[junit4] 2> 2891924 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@4da2fbd3[tmp_collection_shard1_replica_n41] main]
[junit4] 2> 2891925 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2891926 INFO (qtp1092877273-123706) [n:127.0.0.1:42516_ c:tmp_collection s:shard1 x:tmp_collection_shard1_replica_n41] o.a.s.r.Manage
[...truncated too long message...]
onfigure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null890186909
[copy] Copying 238 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null890186909
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null890186909
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 165ms :: artifacts dl 4ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 300 minutes 49 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 23 -
Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/23/
16 tests failed.
FAILED: org.apache.lucene.document.TestLatLonLineShapeQueries.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([BE6FBD2BC96F66EA:3938C0A458361A6A]:0)
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:215)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:186)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:144)
at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4436)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4058)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2160)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1993)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1944)
at org.apache.lucene.document.BaseLatLonShapeTestCase.indexRandomShapes(BaseLatLonShapeTestCase.java:226)
at org.apache.lucene.document.BaseLatLonShapeTestCase.verify(BaseLatLonShapeTestCase.java:192)
at org.apache.lucene.document.BaseLatLonShapeTestCase.doTestRandom(BaseLatLonShapeTestCase.java:173)
at org.apache.lucene.document.BaseLatLonShapeTestCase.testRandomBig(BaseLatLonShapeTestCase.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:45887/_evt/zi/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:45887/_evt/zi/collection1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:7B998E8E5A98AB58]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1591)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 53 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
Error Message:
4 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest: 1) Thread[id=81940, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-EventThread, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 2) Thread[id=81941, name=zkConnectionManagerCallback-10504-thread-1, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=81938, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=81939, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-SendThread(127.0.0.1:41152), state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest:
1) Thread[id=81940, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-EventThread, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
2) Thread[id=81941, name=zkConnectionManagerCallback-10504-thread-1, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=81938, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=81939, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-SendThread(127.0.0.1:41152), state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=81941, name=zkConnectionManagerCallback-10504-thread-1, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=81939, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-SendThread(127.0.0.1:41152), state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=81941, name=zkConnectionManagerCallback-10504-thread-1, state=WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=81939, name=TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[F3CDB154F464C6A0]-SendThread(127.0.0.1:41152), state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:7B998E8E5A98AB58]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:921)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1478)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 1) Thread[id=192562, name=searcherExecutor-15806-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=192481, name=searcherExecutor-15786-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=192598, name=searcherExecutor-15820-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=192355, name=searcherExecutor-15744-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=192437, name=searcherExecutor-15772-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=192634, name=searcherExecutor-15834-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=192399, name=searcherExecutor-15758-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:
1) Thread[id=192562, name=searcherExecutor-15806-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=192481, name=searcherExecutor-15786-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=192598, name=searcherExecutor-15820-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=192355, name=searcherExecutor-15744-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=192437, name=searcherExecutor-15772-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=192634, name=searcherExecutor-15834-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=192399, name=searcherExecutor-15758-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=192562, name=searcherExecutor-15806-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=192481, name=searcherExecutor-15786-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=192598, name=searcherExecutor-15820-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=192355, name=searcherExecutor-15744-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=192437, name=searcherExecutor-15772-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=192634, name=searcherExecutor-15834-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=192399, name=searcherExecutor-15758-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=192562, name=searcherExecutor-15806-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=192481, name=searcherExecutor-15786-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=192598, name=searcherExecutor-15820-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=192355, name=searcherExecutor-15744-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=192437, name=searcherExecutor-15772-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=192634, name=searcherExecutor-15834-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=192399, name=searcherExecutor-15758-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0]:0)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:84D7C8F05C7DC78C]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete(CdcrReplicationDistributedZkTest.java:534)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:F387BE63A254BE2A]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader(CdcrReplicationDistributedZkTest.java:441)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:EFB52DB7BB57CD7D]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps(CdcrReplicationDistributedZkTest.java:471)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:213DFDB7AACB6092]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange(CdcrReplicationDistributedZkTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:D1E268018DC00747]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries(CdcrReplicationDistributedZkTest.java:567)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:700E6FC7E0BC4D29]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:AFD056D53A447296]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:53AF23CE243D03E5]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget(CdcrReplicationDistributedZkTest.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:DA2E9F73644E5B1]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation(CdcrReplicationDistributedZkTest.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10056 lines...]
[junit4] Suite: org.apache.lucene.document.TestLatLonLineShapeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLatLonLineShapeQueries -Dtests.method=testRandomBig -Dtests.seed=BE6FBD2BC96F66EA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-SG -Dtests.timezone=America/Campo_Grande -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 344s J2 | TestLatLonLineShapeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([BE6FBD2BC96F66EA:3938C0A458361A6A]:0)
[junit4] > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
[junit4] > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
[junit4] > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
[junit4] > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
[junit4] > at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:215)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
[junit4] > at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
[junit4] > at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
[junit4] > at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:186)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:144)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4436)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4058)
[junit4] > at org.apache.lucene.index.SerialMergeSche
[...truncated too long message...]
t4] 2> 2 /solr/collections/target_collection/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 130 /solr/clusterprops.json
[junit4] 2> 130 /solr/clusterstate.json
[junit4] 2> 74 /solr/collections/tmp_collection/state.json
[junit4] 2> 12 /solr/collections/tmp_collection/cdcr/state/process
[junit4] 2> 12 /solr/collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 5 /solr/collections/source_collection/cdcr/state/process
[junit4] 2> 4 /solr/collections/target_collection/cdcr/state/buffer
[junit4] 2> 4 /solr/collections/source_collection/state.json
[junit4] 2> 4 /solr/collections/source_collection/cdcr/state/buffer
[junit4] 2> 4 /solr/collections/target_collection/cdcr/state/process
[junit4] 2> 4 /solr/collections/target_collection/state.json
[junit4] 2> 3 /solr/autoscaling.json
[junit4] 2> 2 /solr/collections/tmp_collection/leader_elect/shard2/election/72136003509616648-core_node44-n_0000000000
[junit4] 2> 2 /solr/collections/tmp_collection/leader_elect/shard1/election/72136003509616644-core_node42-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 130 /solr/live_nodes
[junit4] 2> 130 /solr/collections
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 3 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 3 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 3 /solr/overseer/queue-work
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CdcrReplicationDistributedZkTest -Dtests.method=testUpdateLogSynchronisation -Dtests.seed=F3CDB154F464C6A0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=pl-PL -Dtests.timezone=America/St_Thomas -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 169s J0 | CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
[junit4] > at __randomizedtesting.SeedInfo.seed([F3CDB154F464C6A0:DA2E9F73644E5B1]:0)
[junit4] > at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
[junit4] > at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation(CdcrReplicationDistributedZkTest.java:382)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrReplicationDistributedZkTest_F3CDB154F464C6A0-001
[junit4] 2> Aug 10, 2018 1:30:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1305, maxMBSortInHeap=7.146423373717825, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6c4a3bea), locale=pl-PL, timezone=America/St_Thomas
[junit4] 2> NOTE: Linux 4.4.0-130-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=105311792,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [DeleteLastCustomShardedReplicaTest, DateFieldTest, TestNumericTerms32, TestImpersonationWithHadoopAuth, ReplaceNodeTest, HighlighterTest, TestTrieFacet, PeerSyncWithIndexFingerprintCachingTest, HttpPartitionTest, BasicDistributedZk2Test, TestRequestStatusCollectionAPI, AliasIntegrationTest, TestGroupingSearch, TestBinaryResponseWriter, OverriddenZkACLAndCredentialsProvidersTest, TestFieldCacheVsDocValues, ShardRoutingCustomTest, DistributedVersionInfoTest, TestPayloadCheckQParserPlugin, LukeRequestHandlerTest, TestStressInPlaceUpdates, TestHdfsCloudBackupRestore, TestSearchPerf, ExternalFileFieldSortTest, TestBackupRepositoryFactory, RequiredFieldsTest, TestManagedStopFilterFactory, TestImplicitCoreProperties, TestFieldCacheReopen, ChaosMonkeySafeLeaderWithPullReplicasTest, TestSchemaVersionResource, RollingRestartTest, SpatialHeatmapFacetsTest, TestRecoveryHdfs, TestWithCollection, TestFunctionQuery, TestSSLRandomization, TestFieldResource, TestXmlQParserPlugin, ManagedSchemaRoundRobinCloudTest, BlockJoinFacetDistribTest, DataDrivenBlockJoinTest, TestSolrConfigHandler, TestChildDocTransformer, MetricsHistoryHandlerTest, TestClusterStateMutator, RecoveryZkTest, OverseerRolesTest, TestCloudConsistency, IndexBasedSpellCheckerTest, TestSolrCoreProperties, SolrIndexMetricsTest, ZkNodePropsTest, TestSortByMinMaxFunction, FullSolrCloudDistribCmdsTest, BufferStoreTest, QueryEqualityTest, SampleTest, TestLuceneMatchVersion, DistributedSpellCheckComponentTest, TestLegacyFieldReuse, ConnectionReuseTest, TestReRankQParserPlugin, DistributedFacetPivotSmallTest, TestSolrCloudWithKerberosAlt, OverseerTest, TestUnInvertedFieldException, TestCoreBackup, SolrSlf4jReporterTest, TestAnalyzedSuggestions, FastVectorHighlighterTest, DocExpirationUpdateProcessorFactoryTest, TermVectorComponentTest, TestPHPSerializedResponseWriter, CoreAdminHandlerTest, TestSolrQueryParser, AddBlockUpdateTest, TestDistribDocBasedVersion, TestRawTransformer, TestPhraseSuggestions, RequestLoggingTest, TestBinaryField, TestNodeAddedTrigger, TestRuleBasedAuthorizationPlugin, BitVectorTest, ConjunctionSolrSpellCheckerTest, ConcurrentCreateRoutedAliasTest, TestLRUStatsCache, GraphQueryTest, TriggerCooldownIntegrationTest, MultiThreadedOCPTest, HdfsAutoAddReplicasIntegrationTest, CreateCollectionCleanupTest, OpenCloseCoreStressTest, PKIAuthenticationIntegrationTest, TestSolrConfigHandlerConcurrent, SubstringBytesRefFilterTest, SoftAutoCommitTest, TestInitQParser, SimpleCollectionCreateDeleteTest, TestRTimerTree, TolerantUpdateProcessorTest, HDFSCollectionsAPITest, SearchRateTriggerTest, CdcrBidirectionalTest, TestCloudJSONFacetJoinDomain, FileUtilsTest, TestFastOutputStream, LeaderFailoverAfterPartitionTest, SolrGraphiteReporterTest, TestCrossCoreJoin, TestExpandComponent, RAMDirectoryFactoryTest, TestSQLHandler, TestTolerantUpdateProcessorCloud, TestEmbeddedSolrServerConstructors, TestScoreJoinQPNoScore, TestSimDistributedQueue, DirectoryFactoryTest, TestIntervalFaceting, SystemInfoHandlerTest, TestDocTermOrds, ReplicaListTransformerTest, TestIBSimilarityFactory, TestMiniSolrCloudClusterSSL, TestRandomDVFaceting, PathHierarchyTokenizerFactoryTest, TestManagedSchemaThreadSafety, TestClassicSimilarityFactory, AutoScalingHandlerTest, TestMultiValuedNumericRangeQuery, CloudMLTQParserTest, TestClusterStateProvider, TestDocBasedVersionConstraints, ChaosMonkeyNothingIsSafeTest, AnalysisAfterCoreReloadTest, BasicFunctionalityTest, ConvertedLegacyTest, CursorPagingTest, EchoParamsTest, MinimalSchemaTest, OutputWriterTest, TestCursorMarkWithoutUniqueKey, TestDistributedGrouping, TestDistributedMissingSort, TestRandomFaceting, TestSimpleTrackingShardHandler, TestTolerantSearch, TestTrie, ProtectedTermFilterFactoryTest, TestCharFilters, TokenizerChainTest, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ActionThrottleTest, AddReplicaTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ChaosMonkeySafeLeaderTest, CleanupOldIndexTest, ConfigSetsAPITest, ConnectionManagerTest, CreateRoutedAliasTest, DeleteNodeTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, DistribJoinFromCollectionTest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderTest, LeaderVoteWaitTimeoutTest, MetricsHistoryIntegrationTest, MoveReplicaHDFSTest, SSLMigrationTest, SaslZkACLProviderTest, SolrCloudExampleTest, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestCloudRecovery, TestCloudSearcherWarming, TestConfigSetsAPIZkFailure, TestExclusionRuleCollectionAccess, TestPullReplica, TestTolerantUpdateProcessorRandomCloud, TestZkChroot, TlogReplayBufferedWhileIndexingTest, HdfsCollectionsAPIDistributedZkTest, AutoAddReplicasPlanActionTest, ExecutePlanActionTest, MetricTriggerIntegrationTest, NodeMarkersRegistrationTest, ScheduledTriggerTest, TestComputePlanAction, TestDistribStateManager, TestExecutePlanAction, TestGenericDistributedQueue, TestLargeCluster, TestTriggerIntegration, BaseCdcrDistributedZkTest, CdcrBootstrapTest, CdcrReplicationDistributedZkTest]
[junit4] Completed [829/829 (3!)] on J0 in 1543.99s, 11 tests, 9 errors <<< FAILURES!
[...truncated 8746 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid9501.hprof
Total time: 319 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 22 -
Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/22/
17 tests failed.
FAILED: org.apache.lucene.index.TestIndexWriterReader.testDuringAddIndexes
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([195B60C59D5CF096:EB820ED6FAF5E600]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.lucene.index.TestIndexWriterReader.testDuringAddIndexes(TestIndexWriterReader.java:772)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.lucene.search.TestApproximationSearchEquivalence.testReqOpt
Error Message:
target = 4043 < docID = 4044
Stack Trace:
java.lang.AssertionError: target = 4043 < docID = 4044
at __randomizedtesting.SeedInfo.seed([195B60C59D5CF096:7DBAE508B63B8854]:0)
at org.apache.lucene.search.AssertingScorer.advanceShallow(AssertingScorer.java:81)
at org.apache.lucene.search.ReqOptSumScorer.advanceShallow(ReqOptSumScorer.java:215)
at org.apache.lucene.search.AssertingScorer.advanceShallow(AssertingScorer.java:82)
at org.apache.lucene.search.BlockMaxConjunctionScorer.advanceShallow(BlockMaxConjunctionScorer.java:228)
at org.apache.lucene.search.BlockMaxConjunctionScorer$1.moveToNextBlock(BlockMaxConjunctionScorer.java:89)
at org.apache.lucene.search.BlockMaxConjunctionScorer$1.advanceTarget(BlockMaxConjunctionScorer.java:110)
at org.apache.lucene.search.BlockMaxConjunctionScorer$1.doNext(BlockMaxConjunctionScorer.java:181)
at org.apache.lucene.search.BlockMaxConjunctionScorer$1.advance(BlockMaxConjunctionScorer.java:137)
at org.apache.lucene.search.BlockMaxConjunctionScorer$1.nextDoc(BlockMaxConjunctionScorer.java:132)
at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:261)
at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:214)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:71)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:652)
at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:72)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:567)
at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
at org.apache.lucene.search.SearchEquivalenceTestBase.assertSameScores(SearchEquivalenceTestBase.java:251)
at org.apache.lucene.search.SearchEquivalenceTestBase.assertSameScores(SearchEquivalenceTestBase.java:228)
at org.apache.lucene.search.TestApproximationSearchEquivalence.testReqOpt(TestApproximationSearchEquivalence.java:296)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([6B7FBFDDC9BD66CF:EC28C25258E41A4F]:0)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1751)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:186)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:144)
at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4436)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4058)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2160)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5093)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1602)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1218)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
FAILED: org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:F7E66AEAE3224770]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:460)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR(LIROnShardRestartTest.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTrigger
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:A980F93EA768A20D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTrigger(TestTriggerIntegration.java:468)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:CEB566AE046D53BA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:2EBD8EBF897AAB1E]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:5311088F5F820386]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation(CdcrReplicationDistributedZkTest.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:AD345F1BCB92581D]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBufferOnNonLeader(CdcrReplicationDistributedZkTest.java:441)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:DA64298835BB21BB]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchAddsWithDelete(CdcrReplicationDistributedZkTest.java:534)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:8F518979E406E170]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testBatchBoundaries(CdcrReplicationDistributedZkTest.java:567)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:F163B7AD538294A1]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:B106CCCFD2912B4A]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testOps(CdcrReplicationDistributedZkTest.java:471)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:7F8E1CCFC30D86A5]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange(CdcrReplicationDistributedZkTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097:D1CC2B64DFBE5D2]:0)
at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:811)
at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget(CdcrReplicationDistributedZkTest.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 570 lines...]
[junit4] Suite: org.apache.lucene.search.TestApproximationSearchEquivalence
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestApproximationSearchEquivalence -Dtests.method=testReqOpt -Dtests.seed=195B60C59D5CF096 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=th-TH -Dtests.timezone=Pacific/Tahiti -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 2.01s J1 | TestApproximationSearchEquivalence.testReqOpt <<<
[junit4] > Throwable #1: java.lang.AssertionError: target = 4043 < docID = 4044
[junit4] > at __randomizedtesting.SeedInfo.seed([195B60C59D5CF096:7DBAE508B63B8854]:0)
[junit4] > at org.apache.lucene.search.AssertingScorer.advanceShallow(AssertingScorer.java:81)
[junit4] > at org.apache.lucene.search.ReqOptSumScorer.advanceShallow(ReqOptSumScorer.java:215)
[junit4] > at org.apache.lucene.search.AssertingScorer.advanceShallow(AssertingScorer.java:82)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer.advanceShallow(BlockMaxConjunctionScorer.java:228)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer$1.moveToNextBlock(BlockMaxConjunctionScorer.java:89)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer$1.advanceTarget(BlockMaxConjunctionScorer.java:110)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer$1.doNext(BlockMaxConjunctionScorer.java:181)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer$1.advance(BlockMaxConjunctionScorer.java:137)
[junit4] > at org.apache.lucene.search.BlockMaxConjunctionScorer$1.nextDoc(BlockMaxConjunctionScorer.java:132)
[junit4] > at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:261)
[junit4] > at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:214)
[junit4] > at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
[junit4] > at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:71)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:652)
[junit4] > at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:72)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:567)
[junit4] > at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
[junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
[junit4] > at org.apache.lucene.search.SearchEquivalenceTestBase.assertSameScores(SearchEquivalenceTestBase.java:251)
[junit4] > at org.apache.lucene.search.SearchEquivalenceTestBase.assertSameScores(SearchEquivalenceTestBase.java:228)
[junit4] > at org.apache.lucene.search.TestApproximationSearchEquivalence.testReqOpt(TestApproximationSearchEquivalence.java:296)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@150ed858), locale=th-TH, timezone=Pacific/Tahiti
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=389119792,total=526385152
[junit4] 2> NOTE: All tests run in this JVM: [TestBoolean2ScorerSupplier, TestNativeFSLockFactory, TestPointValues, TestSegmentReader, TestSubScorerFreqs, TestSoftDeletesRetentionMergePolicy, TestOneMergeWrappingMergePolicy, TestDocValuesFieldUpdates, TestStressDeletes, TestShardSearching, Test2BDocs, TestFrequencyTrackingRingBuffer, TestIntroSelector, TestDoubleRangeFieldQueries, TestLSBRadixSorter, TestLucene50StoredFieldsFormat, TestMatchAllDocsQuery, TestPayloads, TestFilterCodecReader, TestMultiMMap, TestParallelLeafReader, TestConcurrentMergeScheduler, TestReusableStringReader, TestBooleanScorer, TestIndexOptions, TestIndexFileDeleter, TestBooleanQuery, TestBinaryDocument, TestRecyclingByteBlockAllocator, TestStringMSBRadixSorter, TestApproximationSearchEquivalence]
[junit4] Completed [86/503 (1!)] on J1 in 30.27s, 10 tests, 1 failure <<< FAILURES!
[...truncated 278 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexWriterReader
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterReader -Dtests.method=testDuringAddIndexes -Dtests.seed=195B60C59D5CF096 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Asia/Yangon -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 7.95s J2 | TestIndexWriterReader.testDuringAddIndexes <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([195B60C59D5CF096:EB820ED6FAF5E600]:0)
[junit4] > at org.apache.lucene.index.TestIndexWriterReader.testDuringAddIndexes(TestIndexWriterReader.java:772)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 1> TEST: now get reader
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/core/test/J2/temp/lucene.index.TestIndexWriterReader_195B60C59D5CF096-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {field1=FSTOrd50, indexname=PostingsFormat(name=LuceneFixedGap), field=PostingsFormat(name=LuceneVarGapFixedInterval), foo=PostingsFormat(name=LuceneVarGapFixedInterval), id=FSTOrd50, field3=PostingsFormat(name=LuceneFixedGap), field2=PostingsFormat(name=LuceneVarGapFixedInterval), field5=FSTOrd50, field4=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, maxPointsInLeafNode=258, maxMBSortInHeap=7.974682291420739, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7f24c04), locale=de-AT, timezone=Asia/Yangon
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=176866848,total=520617984
[junit4] 2> NOTE: All tests run in this JVM: [TestFloatRangeFieldQueries, TestInPlaceMergeSorter, TestMathUtil, TestDuelingCodecs, TestPostingsOffsets, TestIndexSearcher, TestFilterDirectory, TestInfoStream, TestIndependenceChiSquared, Test2BPostings, TestBlockMaxConjunction, TestMergeRateLimiter, TestScorerPerf, TestSegmentTermDocs, TestSortRandom, TestPackedTokenAttributeImpl, TestMutablePointsReaderUtils, TestOrdinalMap, TestNotDocIdSet, TestReqOptSumScorer, TestBoostQuery, TestStressIndexing, TestMultiCollector, TestBinaryTerms, TestClassicSimilarity, TestBM25Similarity, TestFeatureField, TestSnapshotDeletionPolicy, TestFSTs, TestCustomNorms, TestIndexWriterReader]
[junit4] Completed [147/503 (2!)] on J2 in 153.33s, 23 tests, 1 failure <<< FAILURES!
[...truncated 8584 lines...]
[junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=6B7FBFDDC9BD66CF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-PY -Dtests.timezone=Asia/Saigon -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 1626s J0 | TestInetAddressRangeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] > at __randomizedtesting.SeedInfo.seed([6B7FBFDDC9BD66CF:EC28C25258E41A4F]:0)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1751)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
[junit4] > at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
[junit4] > at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
[junit4] > at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:186)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:144)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4436)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4058)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2160)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5093)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1602)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1218)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] > at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:83)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/misc/test/J0/temp/lucene.search.TestInetAddressRangeQueries_6B7FBFDDC9BD66CF-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneFixedGap)}, docValues:{ipRangeField=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=425, maxMBSortInHeap=7.0253761881456365, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@149ab12c), locale=es-PY, timezone=Asia/Saigon
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=305003784,total=394264576
[junit4] 2> NOTE: All tests run in this JVM: [TestInetAddressPoint, TestInetAddressRangeQueries]
[junit4] Completed [12/13 (1!)] on J0 in 1664.65s, 4 tests, 1 error <<< FAILURES!
[...truncated 4 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/misc/test/temp/junit4-J0-20180803_151928_6342286848303150833793.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/heapdumps/java_pid16187.hprof ...
[junit4] Heap dump file created [561854176 bytes in 6.558 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 4604 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> 3252849 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_AD7E502C9DA22097-001/init-core-data-001
[junit4] 2> 3252850 WARN (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 3252850 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3252868 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3252869 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_AD7E502C9DA22097-001/tempDir-001
[junit4] 2> 3252869 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3252869 INFO (Thread-9428) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3252869 INFO (Thread-9428) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3252879 ERROR (Thread-9428) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3252969 INFO (SUITE-IndexSizeTriggerTest-seed#[AD7E502C9DA22097]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:34925
[junit4] 2> 3253006 INFO (zkConnectionManagerCallback-2419-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3253080 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 3253080 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 3253123 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3253123 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3253123 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 3253176 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3253176 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3253176 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 3253199 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7155c899{/solr,null,AVAILABLE}
[junit4] 2> 3253268 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@11dd6d29{/solr,null,AVAILABLE}
[junit4] 2> 3253269 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@a386548{SSL,[ssl, http/1.1]}{127.0.0.1:44349}
[junit4] 2> 3253269 INFO (jetty-launcher-2416-thread-1) [ ] o.e.j.s.Server Started @3253332ms
[junit4] 2> 3253269 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44349}
[junit4] 2> 3253284 ERROR (jetty-launcher-2416-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3253284 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3253285 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 3253285 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3253285 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3253285 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-03T18:40:03.763Z
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@293f2f92{SSL,[ssl, http/1.1]}{127.0.0.1:40762}
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.e.j.s.Server Started @3253370ms
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40762}
[junit4] 2> 3253307 ERROR (jetty-launcher-2416-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3253307 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-03T18:40:03.785Z
[junit4] 2> 3253334 INFO (zkConnectionManagerCallback-2421-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3253343 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3253350 INFO (zkConnectionManagerCallback-2423-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3253382 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3253399 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x103a559717d0001, likely client has closed socket
[junit4] 2> 3253819 INFO (jetty-launcher-2416-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34925/solr
[junit4] 2> 3253854 INFO (zkConnectionManagerCallback-2427-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3253907 INFO (zkConnectionManagerCallback-2429-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3254550 INFO (jetty-launcher-2416-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34925/solr
[junit4] 2> 3254552 INFO (zkConnectionManagerCallback-2435-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3254572 INFO (zkConnectionManagerCallback-2437-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3254630 INFO (jetty-launcher-2416-thread-2) [n:127.0.0.1:40762_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40762_solr
[junit4] 2> 3254631 INFO (jetty-launcher-2416-thread-2) [n:127.0.0.1:40762_solr ] o.a.s.c.Overseer Overseer (id=73083822542749702-127.0.0.1:40762_solr-n_0000000000) starting
[junit4] 2> 3254732 INFO (zkConnectionManagerCallback-2444-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3254764 INFO (jetty-laun
[...truncated too long message...]
ingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> ??? 04, 2018 7:50:39 ? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=174650, name=TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[AD7E502C9DA22097], state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestGenericDistributedQueue_AD7E502C9DA22097-001
[junit4] 2> Aug 03, 2018 9:50:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=2019, maxMBSortInHeap=7.743694575072179, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@ec7d939), locale=ar, timezone=Asia/Ust-Nera
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=99784944,total=524812288
[junit4] 2> NOTE: All tests run in this JVM: [TestCustomSort, TestHashPartitioner, TestCorePropertiesReload, ExternalFileFieldSortTest, PeerSyncWithIndexFingerprintCachingTest, TestUseDocValuesAsStored, AddSchemaFieldsUpdateProcessorFactoryTest, CollectionReloadTest, TestManagedSchemaThreadSafety, TestFunctionQuery, TestCryptoKeys, TestTolerantUpdateProcessorRandomCloud, TestSolrCoreParser, DateFieldTest, ChangedSchemaMergeTest, TestCloudSchemaless, CursorMarkTest, TestInitQParser, TestSolrDeletionPolicy1, LoggingHandlerTest, TestDynamicLoading, HdfsThreadLeakTest, TestLeaderInitiatedRecoveryThread, OverseerCollectionConfigSetProcessorTest, ClassificationUpdateProcessorFactoryTest, ExitableDirectoryReaderTest, TestCollectionAPI, MetricsHandlerTest, TestQuerySenderListener, TestNamedUpdateProcessors, UpdateLogTest, TestConfigOverlay, ManagedSchemaRoundRobinCloudTest, MaxSizeAutoCommitTest, TestSegmentSorting, ConjunctionSolrSpellCheckerTest, ConfigSetsAPITest, TestMultiValuedNumericRangeQuery, TestSchemaManager, TestRandomFaceting, TestReRankQParserPlugin, BadComponentTest, TestClassicSimilarityFactory, TestHalfAndHalfDocValues, TestDocTermOrds, BitVectorTest, SSLMigrationTest, TestSystemIdResolver, SecurityConfHandlerTest, TestDocBasedVersionConstraints, TimeZoneUtilsTest, NodeMutatorTest, RequiredFieldsTest, TestDynamicFieldCollectionResource, TestSQLHandlerNonCloud, TestEmbeddedSolrServerSchemaAPI, MinimalSchemaTest, CoreAdminRequestStatusTest, TestSolrCoreSnapshots, TestChildDocTransformer, TestCloudJSONFacetSKG, ZkStateReaderTest, TestLRUStatsCache, SuggesterTest, TermVectorComponentTest, TestZkChroot, SystemLogListenerTest, PreAnalyzedFieldManagedSchemaCloudTest, TestSkipOverseerOperations, SolrGraphiteReporterTest, SpatialFilterTest, DistributedQueryComponentCustomSortTest, MetricsConfigTest, OverseerStatusTest, ClusterStateTest, SolrIndexMetricsTest, TestPHPSerializedResponseWriter, TestCursorMarkWithoutUniqueKey, MetricUtilsTest, TestLazyCores, TestReqParamsAPI, ReplicationFactorTest, TestNodeLostTrigger, SolrInfoBeanTest, NoCacheHeaderTest, TestInfoStreamLogging, TestTrieFacet, TestSearchPerf, SuggestComponentContextFilterQueryTest, TestUnInvertedFieldException, HdfsWriteToMultipleCollectionsTest, BlockJoinFacetDistribTest, RegexBoostProcessorTest, TestMacros, ResponseBuilderTest, TestBinaryResponseWriter, DistributedFacetPivotSmallTest, IndexBasedSpellCheckerTest, TaggerTest, TestStressLiveNodes, TestGeoJSONResponseWriter, DocExpirationUpdateProcessorFactoryTest, DOMUtilTest, LegacyCloudClusterPropTest, CoreAdminCreateDiscoverTest, TestSortByMinMaxFunction, TestTolerantSearch, TestDistributedSearch, TestSystemCollAutoCreate, SpatialHeatmapFacetsTest, TestInPlaceUpdatesDistrib, TestImpersonationWithHadoopAuth, DistribJoinFromCollectionTest, HdfsRecoveryZkTest, DistributedSpellCheckComponentTest, MetricTriggerTest, XmlUpdateRequestHandlerTest, LIROnShardRestartTest, TestSubQueryTransformerDistrib, TestSmileRequest, TestTrackingShardHandlerFactory, TestSolrQueryParser, TestRandomCollapseQParserPlugin, TestLocalFSCloudBackupRestore, TestSolrXml, RestoreTriggerStateTest, TestAnalyzedSuggestions, TestXmlQParserPlugin, TestSlowCompositeReaderWrapper, TestDistribIDF, BufferStoreTest, TestShardHandlerFactory, TestLegacyNumericRangeQueryBuilder, ProtectedTermFilterFactoryTest, TestSchemaVersionResource, DeleteNodeTest, AnalysisAfterCoreReloadTest, BasicFunctionalityTest, CursorPagingTest, DisMaxRequestHandlerTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestGroupingSearch, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestLuceneMatchVersion, TestReversedWildcardFilterFactory, TokenizerChainTest, TestLuceneIndexBackCompat, ConnectionReuseTest, ActionThrottleTest, AliasIntegrationTest, AssignBackwardCompatibilityTest, AsyncCallRequestStatusResponseTest, BasicDistributedZk2Test, BasicDistributedZkTest, TestDistribDocBasedVersion, TestLockTree, TestMiniSolrCloudClusterSSL, TestPullReplicaErrorHandling, TestRebalanceLeaders, TlogReplayBufferedWhileIndexingTest, UnloadDistributedZkTest, ZkSolrClientTest, CollectionsAPIAsyncDistributedZkTest, HdfsCollectionsAPIDistributedZkTest, AutoScalingHandlerTest, ComputePlanActionTest, HdfsAutoAddReplicasIntegrationTest, NodeLostTriggerIntegrationTest, ScheduledMaintenanceTriggerTest, ScheduledTriggerIntegrationTest, TriggerCooldownIntegrationTest, TriggerSetPropertiesIntegrationTest, TestClusterStateProvider, TestComputePlanAction, TestExecutePlanAction, TestGenericDistributedQueue]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.seed=AD7E502C9DA22097 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ar -Dtests.timezone=Asia/Ust-Nera -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestGenericDistributedQueue (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([AD7E502C9DA22097]:0)
[junit4] Completed [827/827 (5!)] on J1 in 7280.43s, 2 tests, 2 errors <<< FAILURES!
[...truncated 8739 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid16187.hprof
Total time: 697 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 21
- Still Failing
Posted by Steve Rowe <sa...@gmail.com>.
Chris Lambertus rebooted lucene1-us-west.a.o, and did some light cleanup on the root partition, but there is still little free space there; in the short-term, though, Jenkins should be functional. At Chris’s request I created https://issues.apache.org/jira/browse/INFRA-16833 to free up more space.
FYI lucene2-us-west.a.o (the second VM used to run ASF Jenkins jobs) did not run out of disk space.
--
Steve
www.lucidworks.com
> On Jul 27, 2018, at 12:46 PM, Steve Rowe <sa...@gmail.com> wrote:
>
> At least one of the lucene Jenkins VMs (lucene1-us-west.apache.org) has run out of disk space. I ran ‘ant clean’ on all Jenkins workspaces. However, the root filesystem (separate from the filesystem hosting Jenkins’s workspaces) has run out of space, and I don’t have permissions to even look at some of it, so I’m going to contact Infra on hipchat to see what they can do.
>
> --
> Steve
> www.lucidworks.com
>
>> On Jul 27, 2018, at 12:20 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:
>>
>> Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/21/
>>
>> No tests ran.
>>
>> Build Log:
>> [...truncated 23 lines...]
>> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
>> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> ... 31 more
>> ERROR: Subversion update failed
>> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
>> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
>> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at hudson.FilePath.act(FilePath.java:1036)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Caused: java.io.IOException
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
>> at hudson.FilePath.act(FilePath.java:1043)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Retrying after 10 seconds
>>> git rev-parse --is-inside-work-tree # timeout=10
>> Fetching changes from the remote Git repository
>>> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
>> Fetching upstream changes from git://git.apache.org/lucene-solr.git
>>> git --version # timeout=10
>>> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
>>> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
>>> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
>> Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
>>> git config core.sparsecheckout # timeout=10
>>> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
>> Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
>> Cleaning workspace
>>> git rev-parse --verify HEAD # timeout=10
>> Resetting working tree
>>> git reset --hard # timeout=10
>>> git clean -fdx # timeout=10
>> Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
>> Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
>> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
>> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> ... 31 more
>> ERROR: Subversion update failed
>> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
>> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
>> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at hudson.FilePath.act(FilePath.java:1036)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Caused: java.io.IOException
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
>> at hudson.FilePath.act(FilePath.java:1043)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Retrying after 10 seconds
>>> git rev-parse --is-inside-work-tree # timeout=10
>> Fetching changes from the remote Git repository
>>> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
>> Fetching upstream changes from git://git.apache.org/lucene-solr.git
>>> git --version # timeout=10
>>> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
>>> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
>>> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
>> Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
>>> git config core.sparsecheckout # timeout=10
>>> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
>> Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
>> Cleaning workspace
>>> git rev-parse --verify HEAD # timeout=10
>> Resetting working tree
>>> git reset --hard # timeout=10
>>> git clean -fdx # timeout=10
>> Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
>> Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
>> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
>> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> ... 31 more
>> ERROR: Subversion update failed
>> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
>> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
>> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
>> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
>> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
>> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
>> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
>> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
>> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
>> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
>> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
>> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
>> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
>> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
>> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
>> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
>> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
>> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
>> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
>> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
>> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
>> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
>> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at hudson.FilePath.act(FilePath.java:1036)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Caused: java.io.IOException
>> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
>> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
>> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
>> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
>> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
>> at hudson.FilePath.act(FilePath.java:1043)
>> at hudson.FilePath.act(FilePath.java:1025)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
>> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
>> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Archiving artifacts
>> Recording test results
>> ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 21
- Still Failing
Posted by Steve Rowe <sa...@gmail.com>.
At least one of the lucene Jenkins VMs (lucene1-us-west.apache.org) has run out of disk space. I ran ‘ant clean’ on all Jenkins workspaces. However, the root filesystem (separate from the filesystem hosting Jenkins’s workspaces) has run out of space, and I don’t have permissions to even look at some of it, so I’m going to contact Infra on hipchat to see what they can do.
--
Steve
www.lucidworks.com
> On Jul 27, 2018, at 12:20 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:
>
> Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/21/
>
> No tests ran.
>
> Build Log:
> [...truncated 23 lines...]
> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> ... 31 more
> ERROR: Subversion update failed
> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at hudson.FilePath.act(FilePath.java:1036)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused: java.io.IOException
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
> at hudson.FilePath.act(FilePath.java:1043)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Retrying after 10 seconds
>> git rev-parse --is-inside-work-tree # timeout=10
> Fetching changes from the remote Git repository
>> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
> Fetching upstream changes from git://git.apache.org/lucene-solr.git
>> git --version # timeout=10
>> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
>> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
>> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
> Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
>> git config core.sparsecheckout # timeout=10
>> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
> Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
> Cleaning workspace
>> git rev-parse --verify HEAD # timeout=10
> Resetting working tree
>> git reset --hard # timeout=10
>> git clean -fdx # timeout=10
> Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
> Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> ... 31 more
> ERROR: Subversion update failed
> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at hudson.FilePath.act(FilePath.java:1036)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused: java.io.IOException
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
> at hudson.FilePath.act(FilePath.java:1043)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Retrying after 10 seconds
>> git rev-parse --is-inside-work-tree # timeout=10
> Fetching changes from the remote Git repository
>> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
> Fetching upstream changes from git://git.apache.org/lucene-solr.git
>> git --version # timeout=10
>> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
>> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
>> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
> Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
>> git config core.sparsecheckout # timeout=10
>> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
> Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
> Cleaning workspace
>> git rev-parse --verify HEAD # timeout=10
> Resetting working tree
>> git reset --hard # timeout=10
>> git clean -fdx # timeout=10
> Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
> Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
> ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
> org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> ... 31 more
> ERROR: Subversion update failed
> org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
> at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
> at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
> at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
> at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
> at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
> at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
> at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
> at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
> at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
> at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
> Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
> at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
> at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
> at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
> at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
> at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
> at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
> at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
> at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
> at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
> Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at hudson.FilePath.act(FilePath.java:1036)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused: java.io.IOException
> at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
> at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
> at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
> at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
> at hudson.FilePath.act(FilePath.java:1043)
> at hudson.FilePath.act(FilePath.java:1025)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
> at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
> at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Archiving artifacts
> Recording test results
> ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 21 -
Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/21/
No tests ran.
Build Log:
[...truncated 23 lines...]
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
... 31 more
ERROR: Subversion update failed
org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.IOException
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
at hudson.FilePath.act(FilePath.java:1043)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
Fetching upstream changes from git://git.apache.org/lucene-solr.git
> git --version # timeout=10
> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
... 31 more
ERROR: Subversion update failed
org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.IOException
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
at hudson.FilePath.act(FilePath.java:1043)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
Fetching upstream changes from git://git.apache.org/lucene-solr.git
> git --version # timeout=10
> git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision d78feb22361cef5323793fdff33de621320d7b4b (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f d78feb22361cef5323793fdff33de621320d7b4b
Commit message: "LUCENE-8429: Avoid stack overflows in DaciukMihovAutomatonBuilder."
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Cleaning up /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
Updating http://svn.apache.org/repos/asf/lucene/test-data at revision '2018-07-27T16:19:00.901 +0000'
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/test-data
org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
... 31 more
ERROR: Subversion update failed
org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$12.runSynchronized(SqlJetEngine.java:535)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runEngineTransaction(SqlJetEngine.java:529)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runTransaction(SqlJetDb.java:238)
at org.tmatesoft.sqljet.core.table.SqlJetDb.runWriteTransaction(SqlJetDb.java:211)
at org.tmatesoft.svn.core.internal.wc17.db.statement.SVNWCDbCreateSchema.exec(SVNWCDbCreateSchema.java:225)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetStatement.done(SVNSqlJetStatement.java:420)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.bumpMovedAway(SVNWCDb.java:5421)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb$BumpRevisionPostUpdate.transaction(SVNWCDb.java:5414)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:258)
at org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.opBumpRevisionPostUpdate(SVNWCDb.java:5366)
at org.tmatesoft.svn.core.internal.wc17.SVNUpdateEditor17.closeEdit(SVNUpdateEditor17.java:1728)
at org.tmatesoft.svn.core.internal.wc.SVNCancellableEditor.closeEdit(SVNCancellableEditor.java:148)
at org.tmatesoft.svn.core.internal.io.dav.handlers.DAVEditorHandler.endElement(DAVEditorHandler.java:492)
at org.tmatesoft.svn.core.internal.io.dav.handlers.BasicDAVHandler.endElement(BasicDAVHandler.java:103)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:898)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.readData(HTTPConnection.java:863)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPRequest.dispatch(HTTPRequest.java:220)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:451)
Caused: org.tmatesoft.svn.core.SVNException: svn: E200030: FULL
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:760)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:897)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:363)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1349)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:847)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.IOException
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:216)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: remote file operation failed: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master at hudson.remoting.Channel@2755bcce:lucene
at hudson.FilePath.act(FilePath.java:1043)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:937)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:864)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:143)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 20 -
Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/20/
20 tests failed.
FAILED: org.apache.lucene.document.TestLatLonShapeQueries.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([ADBF889458EF963A:2AE8F51BC9B6EABA]:0)
at org.apache.lucene.geo.GeoTestUtil.createRegularPolygon(GeoTestUtil.java:325)
at org.apache.lucene.geo.GeoTestUtil.nextPolygon(GeoTestUtil.java:398)
at org.apache.lucene.document.TestLatLonShapeQueries.doTestRandom(TestLatLonShapeQueries.java:127)
at org.apache.lucene.document.TestLatLonShapeQueries.testRandomBig(TestLatLonShapeQueries.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
FAILED: org.apache.solr.cloud.BasicDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:45827/e_j/z/collection1: Async exception during distributed update: Error from server at http://127.0.0.1:54976/e_j/z/collection1_shard1_replica_n47: Bad Request request: http://127.0.0.1:54976/e_j/z/collection1_shard1_replica_n47/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54976%2Fe_j%2Fz%2Fcollection1_shard1_replica_n47%2F&wt=javabin&version=2 Remote error message: Exception writing document id 61 to the index; possible analysis error.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45827/e_j/z/collection1: Async exception during distributed update: Error from server at http://127.0.0.1:54976/e_j/z/collection1_shard1_replica_n47: Bad Request
request: http://127.0.0.1:54976/e_j/z/collection1_shard1_replica_n47/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54976%2Fe_j%2Fz%2Fcollection1_shard1_replica_n47%2F&wt=javabin&version=2
Remote error message: Exception writing document id 61 to the index; possible analysis error.
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:D7B8477E271A0A31]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:509)
at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:687)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:381)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:3C274E26102914E4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration
Error Message:
did not finish processing in time
Stack Trace:
java.lang.AssertionError: did not finish processing in time
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:C553A146BF7F233]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:429)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:E0F9B65A0A0C024F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:525)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:378)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest
Error Message:
18 threads leaked from SUITE scope at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest: 1) Thread[id=37265, name=cdcr-replicator-10107-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=37453, name=cdcr-replicator-10107-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=31698, name=cdcr-replicator-9122-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=37264, name=cdcr-scheduler-10106-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=31713, name=cdcr-replicator-9122-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=37461, name=cdcr-replicator-10107-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=31605, name=cdcr-replicator-9122-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=37417, name=cdcr-replicator-10107-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=31494, name=cdcr-replicator-9122-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=31492, name=cdcr-scheduler-9121-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=37483, name=cdcr-replicator-10107-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=31714, name=cdcr-replicator-9122-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=37462, name=cdcr-replicator-10107-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=31667, name=cdcr-replicator-9122-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=31727, name=cdcr-replicator-9122-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=31706, name=cdcr-replicator-9122-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=37469, name=cdcr-replicator-10107-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=37379, name=cdcr-replicator-10107-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 18 threads leaked from SUITE scope at org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest:
1) Thread[id=37265, name=cdcr-replicator-10107-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=37453, name=cdcr-replicator-10107-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=31698, name=cdcr-replicator-9122-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=37264, name=cdcr-scheduler-10106-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=31713, name=cdcr-replicator-9122-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=37461, name=cdcr-replicator-10107-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=31605, name=cdcr-replicator-9122-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=37417, name=cdcr-replicator-10107-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=31494, name=cdcr-replicator-9122-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=31492, name=cdcr-scheduler-9121-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=37483, name=cdcr-replicator-10107-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=31714, name=cdcr-replicator-9122-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=37462, name=cdcr-replicator-10107-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=31667, name=cdcr-replicator-9122-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=31727, name=cdcr-replicator-9122-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=31706, name=cdcr-replicator-9122-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=37469, name=cdcr-replicator-10107-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=37379, name=cdcr-replicator-10107-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.cdcr.CdcrReplicationDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=37265, name=cdcr-replicator-10107-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=37453, name=cdcr-replicator-10107-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=31698, name=cdcr-replicator-9122-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=37264, name=cdcr-scheduler-10106-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=31713, name=cdcr-replicator-9122-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=37461, name=cdcr-replicator-10107-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=31605, name=cdcr-replicator-9122-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=37417, name=cdcr-replicator-10107-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=31494, name=cdcr-replicator-9122-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=31492, name=cdcr-scheduler-9121-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=37483, name=cdcr-replicator-10107-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=31714, name=cdcr-replicator-9122-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=37462, name=cdcr-replicator-10107-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=31667, name=cdcr-replicator-9122-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=31727, name=cdcr-replicator-9122-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=31706, name=cdcr-replicator-9122-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=37469, name=cdcr-replicator-10107-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=37379, name=cdcr-replicator-10107-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=37265, name=cdcr-replicator-10107-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=37453, name=cdcr-replicator-10107-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=31698, name=cdcr-replicator-9122-thread-4, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=37264, name=cdcr-scheduler-10106-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=31713, name=cdcr-replicator-9122-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=37461, name=cdcr-replicator-10107-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=31605, name=cdcr-replicator-9122-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=37417, name=cdcr-replicator-10107-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=31494, name=cdcr-replicator-9122-thread-1, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=31492, name=cdcr-scheduler-9121-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=37483, name=cdcr-replicator-10107-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=31714, name=cdcr-replicator-9122-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=37462, name=cdcr-replicator-10107-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=31667, name=cdcr-replicator-9122-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=31727, name=cdcr-replicator-9122-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=31706, name=cdcr-replicator-9122-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=37469, name=cdcr-replicator-10107-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=37379, name=cdcr-replicator-10107-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:59449: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59449: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:D7B8477E271A0A31]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.BasicDistributedZkTest.testANewCollectionInOneInstanceWithManualShardAssignement(BasicDistributedZkTest.java:806)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:378)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest: 1) Thread[id=4084, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=4076, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
1) Thread[id=4084, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=4076, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9]:0)
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:37726/oaa/tq/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:37726/oaa/tq/collection1
at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9:D7B8477E271A0A31]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1591)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.docValuesUpdateTest(TestInPlaceUpdatesDistrib.java:377)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistribut
[...truncated too long message...]
pport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 10) Thread[id=31492, name=cdcr-scheduler-9121-thread-1, state=TIMED_WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 11) Thread[id=37483, name=cdcr-replicator-10107-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 12) Thread[id=31714, name=cdcr-replicator-9122-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 13) Thread[id=37462, name=cdcr-replicator-10107-thread-6, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 14) Thread[id=31667, name=cdcr-replicator-9122-thread-3, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 15) Thread[id=31727, name=cdcr-replicator-9122-thread-8, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 16) Thread[id=31706, name=cdcr-replicator-9122-thread-5, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 17) Thread[id=37469, name=cdcr-replicator-10107-thread-7, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 18) Thread[id=37379, name=cdcr-replicator-10107-thread-2, state=WAITING, group=TGRP-CdcrReplicationDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([5FEC78A489E667C9]:0)
[junit4] Completed [746/824 (7!)] on J2 in 1670.90s, 11 tests, 11 errors <<< FAILURES!
[...truncated 8953 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid28560.hprof
Total time: 374 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 19 -
Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/19/
15 tests failed.
FAILED: org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit
Error Message:
this IndexWriter is closed
Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:671)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:685)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3412)
at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([EBB758A2DF516552]:0)
at java.util.HashMap.resize(HashMap.java:704)
at java.util.HashMap.putVal(HashMap.java:663)
at java.util.HashMap.put(HashMap.java:612)
at org.apache.lucene.index.BufferedUpdates.addTerm(BufferedUpdates.java:227)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$TermNode.apply(DocumentsWriterDeleteQueue.java:361)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$DeleteSlice.apply(DocumentsWriterDeleteQueue.java:290)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:318)
at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1284)
at org.apache.lucene.index.TestIndexingSequenceNumbers$2.run(TestIndexingSequenceNumbers.java:206)
FAILED: org.apache.solr.cloud.TestAuthenticationFramework.testBasics
Error Message:
Error from server at http://127.0.0.1:43842/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:43842/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([D9B53FE80F5C4E06:E46D91C437B21076]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43842/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTlogReplica
Error Message:
ObjectTracker found 5 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1045) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:358) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:737) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:507) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:954) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:768) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1045)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:358)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:737)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:507)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:954)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:768)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:869)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([D9B53FE80F5C4E06]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:304)
at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple
Error Message:
IOException occured when talking to server at: http://127.0.0.1:39527/solr
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:39527/solr
at __randomizedtesting.SeedInfo.seed([D9B53FE80F5C4E06:E1061B1628AF9AD7]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple(AutoAddReplicasPlanActionTest.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:39527 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 48 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
events: [CapturedEvent{timestamp=31873332125585496, stage=STARTED, actionName='null', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333066759596, stage=BEFORE_ACTION, actionName='compute_plan', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333111939296, stage=AFTER_ACTION, actionName='compute_plan', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333126165396, stage=BEFORE_ACTION, actionName='execute_plan', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873340302206396, stage=AFTER_ACTION, actionName='execute_plan', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873340324056146, stage=SUCCEEDED, actionName='null', event={ "id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y", "source":"index_size_trigger2", "eventTime":31873329026418046, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":31873332111787796, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873351178574896, stage=STARTED, actionName='null', event={ "id":"713ca3968fd4beT3z4wnpsyqh47sw51fs3wn1k41", "source":"index_size_trigger2", "eventTime":31873345671648446, "eventType":"INDEXSIZE", "properties":{ "__start__":2, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":16240,\n \"node_name\":\"127.0.0.1:10129_solr\",\n \"base_url\":\"http://127.0.0.1:10129/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":16240,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":1.5124678611755371E-5,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":26980,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"base_url\":\"http://127.0.0.1:10130/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":26980,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.512708306312561E-5,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":10240,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":0,\n \"__bytes__\":10240,\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"__docs__\":0,\n \"violationType\":\"belowDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "_enqueue_time_":31873351165883146, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873351334369796, stage=BEFORE_ACTION, actionName='compute_plan', event={ "id":"713ca3968fd4beT3z4wnpsyqh47sw51fs3wn1k41", "source":"index_size_trigger2", "eventTime":31873345671648446, "eventType":"INDEXSIZE", "properties":{ "__start__":2, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":16240,\n \"node_name\":\"127.0.0.1:10129_solr\",\n \"base_url\":\"http://127.0.0.1:10129/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":16240,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":1.5124678611755371E-5,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":26980,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"base_url\":\"http://127.0.0.1:10130/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":26980,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.512708306312561E-5,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":10240,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":0,\n \"__bytes__\":10240,\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"__docs__\":0,\n \"violationType\":\"belowDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "_enqueue_time_":31873351165883146, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873351449577596, stage=AFTER_ACTION, actionName='compute_plan', event={ "id":"713ca3968fd4beT3z4wnpsyqh47sw51fs3wn1k41", "source":"index_size_trigger2", "eventTime":31873345671648446, "eventType":"INDEXSIZE", "properties":{ "__start__":2, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":16240,\n \"node_name\":\"127.0.0.1:10129_solr\",\n \"base_url\":\"http://127.0.0.1:10129/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":16240,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":1.5124678611755371E-5,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":26980,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"base_url\":\"http://127.0.0.1:10130/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":26980,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.512708306312561E-5,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":10240,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":0,\n \"__bytes__\":10240,\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"__docs__\":0,\n \"violationType\":\"belowDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "_enqueue_time_":31873351165883146, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873351473296596, stage=BEFORE_ACTION, actionName='execute_plan', event={ "id":"713ca3968fd4beT3z4wnpsyqh47sw51fs3wn1k41", "source":"index_size_trigger2", "eventTime":31873345671648446, "eventType":"INDEXSIZE", "properties":{ "__start__":2, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":16240,\n \"node_name\":\"127.0.0.1:10129_solr\",\n \"base_url\":\"http://127.0.0.1:10129/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":16240,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":1.5124678611755371E-5,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":26980,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"base_url\":\"http://127.0.0.1:10130/solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":26980,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.512708306312561E-5,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":10240,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":0,\n \"__bytes__\":10240,\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"__docs__\":0,\n \"violationType\":\"belowDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "_enqueue_time_":31873351165883146, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}] expected:<6> but was:<10>
Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=31873332125585496, stage=STARTED, actionName='null', event={
"id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y",
"source":"index_size_trigger2",
"eventTime":31873329026418046,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":31873332111787796,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333066759596, stage=BEFORE_ACTION, actionName='compute_plan', event={
"id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y",
"source":"index_size_trigger2",
"eventTime":31873329026418046,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":31873332111787796,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333111939296, stage=AFTER_ACTION, actionName='compute_plan', event={
"id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y",
"source":"index_size_trigger2",
"eventTime":31873329026418046,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":31873332111787796,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873333126165396, stage=BEFORE_ACTION, actionName='execute_plan', event={
"id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y",
"source":"index_size_trigger2",
"eventTime":31873329026418046,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":31873332111787796,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=31873340302206396, stage=AFTER_ACTION, actionName='execute_plan', event={
"id":"713c9fb66dc57eT3z4wnpsyqh47sw51fs3wn1k3y",
"source":"index_size_trigger2",
"eventTime":31873329026418046,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":22740,\n \"node_name\":\"127.0.0.1:10130_solr\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":22740,\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"state\":\"active\",\n \"INDEX.sizeInGB\":2.117827534675598E-5,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":31873332111787796,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operati
[...truncated too long message...]
ave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_172/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.6.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J1-20180713_201559_8064573147141084456649.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J1-20180713_201559_8068425654393208351823.suites -stdin
[junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1540)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:577: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build.xml:267: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/common-build.xml:1568: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/common-build.xml:1092: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
Total time: 796 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-BadApples-NightlyTests-master #18 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 18 -
Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/18/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 1) Thread[id=2244, name=searcherExecutor-694-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2457, name=searcherExecutor-757-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2493, name=searcherExecutor-771-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=2370, name=searcherExecutor-736-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2287, name=searcherExecutor-708-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=2533, name=searcherExecutor-785-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2325, name=searcherExecutor-722-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:
1) Thread[id=2244, name=searcherExecutor-694-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2457, name=searcherExecutor-757-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2493, name=searcherExecutor-771-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=2370, name=searcherExecutor-736-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=2287, name=searcherExecutor-708-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=2533, name=searcherExecutor-785-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=2325, name=searcherExecutor-722-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([AED86BF7D13E1839]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2244, name=searcherExecutor-694-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2457, name=searcherExecutor-757-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2493, name=searcherExecutor-771-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=2370, name=searcherExecutor-736-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2287, name=searcherExecutor-708-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=2533, name=searcherExecutor-785-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2325, name=searcherExecutor-722-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2244, name=searcherExecutor-694-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2457, name=searcherExecutor-757-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2493, name=searcherExecutor-771-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=2370, name=searcherExecutor-736-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=2287, name=searcherExecutor-708-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=2533, name=searcherExecutor-785-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=2325, name=searcherExecutor-722-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([AED86BF7D13E1839]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([AED86BF7D13E1839:268C542D7FC275C1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:921)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1478)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13072 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/init-core-data-001
[junit4] 2> 247408 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 247408 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 247411 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 247411 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 247412 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ysf/z
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 247739 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 247935 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 247944 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 247988 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59294_hdfs____.79zgz1/webapp
[junit4] 2> 248718 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59294
[junit4] 2> 249328 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 249329 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 249339 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_44123_datanode____.qzaqnb/webapp
[junit4] 2> 249924 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44123
[junit4] 2> 250502 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 250504 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 250562 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42324_datanode____yx1sda/webapp
[junit4] 2> 250887 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:47409) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 250979 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6ee7e880abf432: from storage DS-bc468af8-61a5-4598-8bde-25711ae147a4 node DatanodeRegistration(127.0.0.1:50186, datanodeUuid=fd24de3b-f3df-40a1-9ef4-daa3ed3f23e7, infoPort=49210, infoSecurePort=0, ipcPort=59664, storageInfo=lv=-56;cid=testClusterID;nsid=1752092449;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 250979 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6ee7e880abf432: from storage DS-6975d5ce-c080-4ed6-a2f2-991289102ec6 node DatanodeRegistration(127.0.0.1:50186, datanodeUuid=fd24de3b-f3df-40a1-9ef4-daa3ed3f23e7, infoPort=49210, infoSecurePort=0, ipcPort=59664, storageInfo=lv=-56;cid=testClusterID;nsid=1752092449;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 251571 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[AED86BF7D13E1839]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42324
[junit4] 2> 252559 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:47409) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 252644 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6ee7e8e37564e4: from storage DS-1f24fd73-1641-43f7-803c-a277132ef937 node DatanodeRegistration(127.0.0.1:41524, datanodeUuid=5c101619-96e2-4ba9-bcc7-6862ae77f1d4, infoPort=52362, infoSecurePort=0, ipcPort=56007, storageInfo=lv=-56;cid=testClusterID;nsid=1752092449;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 252644 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6ee7e8e37564e4: from storage DS-093428d9-b825-4a7e-9991-8284f3b3dc69 node DatanodeRegistration(127.0.0.1:41524, datanodeUuid=5c101619-96e2-4ba9-bcc7-6862ae77f1d4, infoPort=52362, infoSecurePort=0, ipcPort=56007, storageInfo=lv=-56;cid=testClusterID;nsid=1752092449;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 252777 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 252802 INFO (Thread-486) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 252802 INFO (Thread-486) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 252817 ERROR (Thread-486) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 252908 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ZkTestServer start zk server on port:39538
[junit4] 2> 252946 INFO (zkConnectionManagerCallback-528-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 252990 INFO (zkConnectionManagerCallback-530-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 252997 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 253003 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 253015 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 253016 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 253019 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 253020 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 253021 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 253022 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 253023 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 253023 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 253039 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 253081 INFO (zkConnectionManagerCallback-534-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 253095 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 253974 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 254026 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 254026 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 254026 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 254032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d2b78c{/ysf/z,null,AVAILABLE}
[junit4] 2> 254032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41568fe7{SSL,[ssl, http/1.1]}{127.0.0.1:60324}
[junit4] 2> 254032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server Started @254126ms
[junit4] 2> 254032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:47409/hdfs__localhost_47409__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-BadApples-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001_tempDir-002_control_data, replicaType=NRT, hostContext=/ysf/z, hostPort=60324, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/control-001/cores}
[junit4] 2> 254033 ERROR (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 254033 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 254033 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 254033 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 254033 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 254033 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-07-07T12:08:59.855Z
[junit4] 2> 254054 INFO (zkConnectionManagerCallback-536-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 254056 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 254056 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/control-001/solr.xml
[junit4] 2> 254059 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 254059 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 254060 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 254530 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39538/solr
[junit4] 2> 254532 INFO (zkConnectionManagerCallback-540-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 254545 INFO (zkConnectionManagerCallback-542-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 254733 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 254734 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60324_ysf%2Fz
[junit4] 2> 254735 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.Overseer Overseer (id=74103446753968133-127.0.0.1:60324_ysf%2Fz-n_0000000000) starting
[junit4] 2> 254813 INFO (zkConnectionManagerCallback-549-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 254815 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39538/solr ready
[junit4] 2> 254853 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60324_ysf%2Fz
[junit4] 2> 254873 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 254879 INFO (OverseerStateUpdate-74103446753968133-127.0.0.1:60324_ysf%2Fz-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 254990 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 255015 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 255032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 255032 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 255038 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/control-001/cores
[junit4] 2> 255138 INFO (zkConnectionManagerCallback-555-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 255139 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 255140 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39538/solr ready
[junit4] 2> 255370 INFO (qtp55101524-2199) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60324_ysf%252Fz&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 255419 INFO (OverseerThreadFactory-689-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 255750 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 255751 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 256803 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 256827 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 257044 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 257135 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 257136 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 257136 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 257136 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 257148 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@537c94fc
[junit4] 2> 257268 WARN (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 257694 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 257694 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 257695 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 30000ms;
[junit4] 2> 257695 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 3000ms;
[junit4] 2> 257697 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=25.76953125, floorSegmentMB=0.4404296875, forceMergeDeletesPctAllowed=8.192828633514441, segmentsPerTier=2.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3231970369394789, reclaimDeletesWeight=1.9646118900367338
[junit4] 2> 257727 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@f8bff9f[control_collection_shard1_replica_n1] main]
[junit4] 2> 257728 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 257728 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 257729 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 257750 INFO (searcherExecutor-694-thread-1-processing-n:127.0.0.1:60324_ysf%2Fz x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@f8bff9f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 257750 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605333516101353472
[junit4] 2> 257987 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 258003 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 258003 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 258003 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:60324/ysf/z/control_collection_shard1_replica_n1/
[junit4] 2> 258003 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 258014 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:60324/ysf/z/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 258014 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 258031 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:60324/ysf/z/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 258033 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 258040 INFO (qtp55101524-2204) [n:127.0.0.1:60324_ysf%2Fz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2290
[junit4] 2> 258047 INFO (qtp55101524-2199) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 258147 INFO (zkCallback-541-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 259048 INFO (qtp55101524-2199) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60324_ysf%252Fz&wt=javabin&version=2} status=0 QTime=3677
[junit4] 2> 259086 INFO (zkConnectionManagerCallback-560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 259087 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 259088 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39538/solr ready
[junit4] 2> 259088 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 259094 INFO (qtp55101524-2202) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 259125 INFO (OverseerCollectionConfigSetProcessor-74103446753968133-127.0.0.1:60324_ysf%2Fz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 259125 INFO (OverseerThreadFactory-689-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 259127 WARN (OverseerThreadFactory-689-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 259271 INFO (qtp55101524-2202) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 259272 INFO (OverseerCollectionConfigSetProcessor-74103446753968133-127.0.0.1:60324_ysf%2Fz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 259273 INFO (qtp55101524-2202) [n:127.0.0.1:60324_ysf%2Fz ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=178
[junit4] 2> 259637 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001 of type NRT
[junit4] 2> 259643 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 259684 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 259684 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 259684 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 259684 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a0d62a{/ysf/z,null,AVAILABLE}
[junit4] 2> 259685 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1754d91e{SSL,[ssl, http/1.1]}{127.0.0.1:37598}
[junit4] 2> 259685 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server Started @259779ms
[junit4] 2> 259685 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:47409/hdfs__localhost_47409__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-BadApples-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/ysf/z, hostPort=37598, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001/cores}
[junit4] 2> 259686 ERROR (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 259686 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 259686 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 259686 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 259686 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 259686 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-07-07T12:09:05.508Z
[junit4] 2> 259734 INFO (zkConnectionManagerCallback-562-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 259739 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 259739 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001/solr.xml
[junit4] 2> 259742 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 259742 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 259743 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 260051 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39538/solr
[junit4] 2> 260072 INFO (zkConnectionManagerCallback-566-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 260107 INFO (zkConnectionManagerCallback-568-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 260112 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 260113 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 260115 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 260115 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37598_ysf%2Fz
[junit4] 2> 260116 INFO (zkCallback-541-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 260146 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 260147 INFO (zkCallback-559-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 260147 INFO (zkCallback-567-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 260172 INFO (zkConnectionManagerCallback-575-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 260173 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 260173 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39538/solr ready
[junit4] 2> 260174 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 260188 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 260196 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 260196 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 260197 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001/cores
[junit4] 2> 260319 INFO (qtp124649433-2254) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:37598_ysf%252Fz&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 260363 INFO (OverseerThreadFactory-689-thread-3) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:37598_ysf%2Fz for creating new replica
[junit4] 2> 260379 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 261443 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 261560 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 261961 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 261998 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 262000 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 262000 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 262000 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 262007 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7a3d677b
[junit4] 2> 262036 WARN (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 262175 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 262175 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 262178 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 30000ms;
[junit4] 2> 262178 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 3000ms;
[junit4] 2> 262181 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=25.76953125, floorSegmentMB=0.4404296875, forceMergeDeletesPctAllowed=8.192828633514441, segmentsPerTier=2.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3231970369394789, reclaimDeletesWeight=1.9646118900367338
[junit4] 2> 262189 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@4d832918[collection1_shard1_replica_n21] main]
[junit4] 2> 262194 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 262196 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 262197 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 262201 INFO (searcherExecutor-708-thread-1-processing-n:127.0.0.1:37598_ysf%2Fz x:collection1_shard1_replica_n21 c:collection1 s:shard1) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@4d832918[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 262204 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605333520771710976
[junit4] 2> 262290 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37598/ysf/z/collection1_shard1_replica_n21/
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy https://127.0.0.1:37598/ysf/z/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 262292 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 262295 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37598/ysf/z/collection1_shard1_replica_n21/ shard1
[junit4] 2> 262502 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 262504 INFO (qtp124649433-2253) [n:127.0.0.1:37598_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2125
[junit4] 2> 262515 INFO (qtp124649433-2254) [n:127.0.0.1:37598_ysf%2Fz c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:37598_ysf%252Fz&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2195
[junit4] 2> 262604 INFO (zkCallback-567-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 263026 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-2-001 of type NRT
[junit4] 2> 263028 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 263061 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 263061 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 263061 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 263061 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cd2d507{/ysf/z,null,AVAILABLE}
[junit4] 2> 263062 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@73777ed3{SSL,[ssl, http/1.1]}{127.0.0.1:43854}
[junit4] 2> 263062 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.e.j.s.Server Started @263155ms
[junit4] 2> 263062 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:47409/hdfs__localhost_47409__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-BadApples-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/ysf/z, hostPort=43854, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-2-001/cores}
[junit4] 2> 263062 ERROR (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 263072 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 263072 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 263072 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 263072 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 263072 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-07-07T12:09:08.894Z
[junit4] 2> 263084 INFO (zkConnectionManagerCallback-578-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 263085 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 263085 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-2-001/solr.xml
[junit4] 2> 263091 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 263091 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 263221 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 263504 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39538/solr
[junit4] 2> 263545 INFO (zkConnectionManagerCallback-582-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 263566 INFO (zkConnectionManagerCallback-584-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 263583 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 263585 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 263587 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 263587 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43854_ysf%2Fz
[junit4] 2> 263589 INFO (zkCallback-567-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263589 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263589 INFO (zkCallback-541-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263589 INFO (zkCallback-559-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263607 INFO (zkCallback-574-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263607 INFO (zkCallback-583-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 263611 INFO (zkConnectionManagerCallback-591-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 263612 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 263613 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39538/solr ready
[junit4] 2> 263613 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 263629 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 263637 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 263637 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cea45d7
[junit4] 2> 263643 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[AED86BF7D13E1839]) [n:127.0.0.1:43854_ysf%2Fz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_AED86BF7D13E1839-001/shard-2-001/cores
[junit4] 2> 263715 INFO (qtp124649433-2257) [n:127.0.0.1:37598_ysf%2Fz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43854_ysf%252Fz&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 263731 INFO (OverseerCollectionConfigSetProcessor-74103446753968133-127.0.0.1:60324_ysf%2Fz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 263733 INFO (OverseerThreadFactory-689-thread-4) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:43854_ysf%2Fz for creating new replica
[junit4] 2> 263921 INFO (qtp359212001-2295) [n:127.0.0.1:43854_ysf%2Fz x:collection1_shard1_replica_n23] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 264046 INFO (zkCallback-567-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 265012 INFO (qtp359212001-2295) [n:127.0.0.1:43854_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 265057 INFO (qtp359212001-2295) [n:127.0.0.1:43854_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 265262 INFO (qtp359212001-2295) [n:127.0.0.1:43854_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 265320 INFO (qtp359212001-2295) [n:127.0.0.1:43854_ysf%2Fz c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, truste
[...truncated too long message...]
nfigure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null169574922
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null169574922
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null169574922
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 101ms :: artifacts dl 14ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 324 minutes 16 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 17 -
Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/17/
19 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestDynamicLoading
Error Message:
76 threads leaked from SUITE scope at org.apache.solr.core.TestDynamicLoading: 1) Thread[id=19156, name=Scheduler-534986721, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=19135, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 3) Thread[id=19168, name=zkConnectionManagerCallback-3809-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=19188, name=qtp323133865-19188, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$281/106803555.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=19192, name=qtp323133865-19192, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=19173, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 7) Thread[id=19128, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 8) Thread[id=19171, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=19196, name=Scheduler-231974071, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=19141, name=Scheduler-1786609889, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=19202, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=19217, name=Thread-7061, state=WAITING, group=TGRP-TestDynamicLoading] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 13) Thread[id=19172, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 14) Thread[id=19133, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=19200, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=19215, name=SolrRrdBackendFactory-5468-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=19131, name=zkCallback-3792-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=19175, name=SolrRrdBackendFactory-5454-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=19147, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=19253, name=SolrRrdBackendFactory-5440-thread-2, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=19134, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 22) Thread[id=19187, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=19194, name=qtp323133865-19194, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=19155, name=qtp1084804310-19155, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=19139, name=Thread-7043, state=WAITING, group=TGRP-TestDynamicLoading] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 26) Thread[id=19191, name=qtp323133865-19191, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=19161, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=19117, name=qtp1179340595-19117, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=19115, name=qtp1179340595-19115, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=19206, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 31) Thread[id=19154, name=qtp1084804310-19154, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=19137, name=SolrRrdBackendFactory-5440-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=19255, name=SolrRrdBackendFactory-5468-thread-2, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=19201, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=19207, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 36) Thread[id=19213, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 37) Thread[id=19153, name=qtp1084804310-19153, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=19214, name=zkConnectionManagerCallback-3832-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=19190, name=qtp323133865-19190-acceptor-0@7bda8c15-ServerConnector@1e5b9d7e{SSL,[ssl, http/1.1]}{127.0.0.1:45161}, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 40) Thread[id=19129, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 41) Thread[id=19144, name=searcherExecutor-5443-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 42) Thread[id=19184, name=searcherExecutor-5457-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 43) Thread[id=19152, name=qtp1084804310-19152, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 44) Thread[id=19189, name=qtp323133865-19189, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$281/106803555.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 45) Thread[id=19148, name=qtp1084804310-19148, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 46) Thread[id=19177, name=Thread-7052, state=WAITING, group=TGRP-TestDynamicLoading] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 47) Thread[id=19136, name=zkConnectionManagerCallback-3800-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 48) Thread[id=19195, name=qtp323133865-19195, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 49) Thread[id=19174, name=zkConnectionManagerCallback-3816-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 50) Thread[id=19150, name=qtp1084804310-19150-acceptor-0@8cb784-ServerConnector@2fe13dae{SSL,[ssl, http/1.1]}{127.0.0.1:34432}, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 51) Thread[id=19113, name=qtp1179340595-19113, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 52) Thread[id=19176, name=MetricsHistoryHandler-5455-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 53) Thread[id=19122, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 54) Thread[id=19162, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 55) Thread[id=19151, name=qtp1084804310-19151, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 56) Thread[id=19216, name=MetricsHistoryHandler-5469-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 57) Thread[id=19110, name=qtp1179340595-19110, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 58) Thread[id=19111, name=qtp1179340595-19111, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 59) Thread[id=19149, name=qtp1084804310-19149, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 60) Thread[id=19116, name=qtp1179340595-19116, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 61) Thread[id=19160, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 62) Thread[id=19193, name=qtp323133865-19193, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 63) Thread[id=19130, name=zkConnectionManagerCallback-3793-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 64) Thread[id=19254, name=SolrRrdBackendFactory-5454-thread-2, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 65) Thread[id=19211, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 66) Thread[id=19124, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 67) Thread[id=19138, name=MetricsHistoryHandler-5441-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 68) Thread[id=19212, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 69) Thread[id=19118, name=Scheduler-445958845, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 70) Thread[id=19114, name=qtp1179340595-19114, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717) at java.lang.Thread.run(Thread.java:748) 71) Thread[id=19112, name=qtp1179340595-19112-acceptor-0@67d326c4-ServerConnector@3371bca2{SSL,[ssl, http/1.1]}{127.0.0.1:37301}, state=RUNNABLE, group=TGRP-TestDynamicLoading] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:748) 72) Thread[id=19166, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 73) Thread[id=19123, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 74) Thread[id=19208, name=zkConnectionManagerCallback-3825-thread-1, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 75) Thread[id=19167, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 76) Thread[id=19179, name=Scheduler-218877429, state=TIMED_WAITING, group=TGRP-TestDynamicLoading] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 76 threads leaked from SUITE scope at org.apache.solr.core.TestDynamicLoading:
1) Thread[id=19156, name=Scheduler-534986721, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=19135, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
3) Thread[id=19168, name=zkConnectionManagerCallback-3809-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=19188, name=qtp323133865-19188, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$281/106803555.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=19192, name=qtp323133865-19192, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=19173, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
7) Thread[id=19128, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
8) Thread[id=19171, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=19196, name=Scheduler-231974071, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=19141, name=Scheduler-1786609889, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=19202, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=19217, name=Thread-7061, state=WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
13) Thread[id=19172, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
14) Thread[id=19133, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=19200, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=19215, name=SolrRrdBackendFactory-5468-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=19131, name=zkCallback-3792-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=19175, name=SolrRrdBackendFactory-5454-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19) Thread[id=19147, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
20) Thread[id=19253, name=SolrRrdBackendFactory-5440-thread-2, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21) Thread[id=19134, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
22) Thread[id=19187, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
23) Thread[id=19194, name=qtp323133865-19194, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
24) Thread[id=19155, name=qtp1084804310-19155, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
25) Thread[id=19139, name=Thread-7043, state=WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
26) Thread[id=19191, name=qtp323133865-19191, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
27) Thread[id=19161, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
28) Thread[id=19117, name=qtp1179340595-19117, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
29) Thread[id=19115, name=qtp1179340595-19115, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
30) Thread[id=19206, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-SendThread(127.0.0.1:45652), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
31) Thread[id=19154, name=qtp1084804310-19154, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
32) Thread[id=19137, name=SolrRrdBackendFactory-5440-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
33) Thread[id=19255, name=SolrRrdBackendFactory-5468-thread-2, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
34) Thread[id=19201, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
35) Thread[id=19207, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
36) Thread[id=19213, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
37) Thread[id=19153, name=qtp1084804310-19153, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
38) Thread[id=19214, name=zkConnectionManagerCallback-3832-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
39) Thread[id=19190, name=qtp323133865-19190-acceptor-0@7bda8c15-ServerConnector@1e5b9d7e{SSL,[ssl, http/1.1]}{127.0.0.1:45161}, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
40) Thread[id=19129, name=TEST-TestDynamicLoading.testDynamicLoading-seed#[F5FFEC26479F8168]-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
41) Thread[id=19144, name=searcherExecutor-5443-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
42) Thread[id=19184, name=searcherExecutor-5457-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
43) Thread[id=19152, name=qtp1084804310-19152, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
44) Thread[id=19189, name=qtp323133865-19189, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$281/106803555.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
45) Thread[id=19148, name=qtp1084804310-19148, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
46) Thread[id=19177, name=Thread-7052, state=WAITING, group=TGRP-TestDynamicLoading]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
47) Thread[id=19136, name=zkConnectionManagerCallback-3800-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
48) Thread[id=19195, name=qtp323133865-19195, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
at java.lang.Thread.run(Thread.java:748)
49) Thread[id=19174, name=zkConnectionManagerCallback-3816-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
50) Thread[id=19150, name=qtp1084804310-19150-acceptor-0@8cb784-ServerConnector@2fe13dae{SSL,[ssl, http/1.1]}{127.0.0.1:34432}, state=RUNNABLE, group=TGRP-TestDynamicLoading]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
51) Thread[id=19113, name=qtp1179340595-19113, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(Lo
[...truncated too long message...]
index.StandardDirectoryReader.open(StandardDirectoryReader.java:105)
[junit4] 2> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:523)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:398)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:332)
[junit4] 2> at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:67)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {field=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=1175, maxMBSortInHeap=6.367309795427806, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@66564222), locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=Asia/Yangon
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=183892920,total=534249472
[junit4] 2> NOTE: All tests run in this JVM: [TestRetrieveFieldsOptimizer, ShardSplitTest, AutoAddReplicasPlanActionTest, SearchRateTriggerIntegrationTest, DirectSolrSpellCheckerTest, HdfsBasicDistributedZkTest, TestPseudoReturnFields, TermsComponentTest, DeleteLastCustomShardedReplicaTest, SolrShardReporterTest, SolrGangliaReporterTest, TestSystemIdResolver, TestSolrCoreProperties, AnalysisAfterCoreReloadTest, TestSkipOverseerOperations, BlockJoinFacetRandomTest, TimeZoneUtilsTest, TestStreamBody, FullSolrCloudDistribCmdsTest, SegmentsInfoRequestHandlerTest, TestDistributedMap, NodeAddedTriggerIntegrationTest, AliasIntegrationTest, TestPolicyCloud, TestNumericRangeQuery64, DistributedQueryComponentOptimizationTest, DistributedFacetPivotWhiteBoxTest, LegacyCloudClusterPropTest, VersionInfoTest, OpenExchangeRatesOrgProviderTest, ConcurrentCreateRoutedAliasTest, XmlInterpolationTest, SortSpecParsingTest, DistributedFacetPivotLongTailTest, ChaosMonkeyNothingIsSafeTest, TestXIncludeConfig, TlogReplayBufferedWhileIndexingTest, DistributedVersionInfoTest, TestSolrCloudWithSecureImpersonation, TestEmbeddedSolrServerConstructors, TestConfigsApi, TestMinMaxOnMultiValuedField, ChaosMonkeySafeLeaderWithPullReplicasTest, TestCloudNestedDocsSort, ReplicationFactorTest, WrapperMergePolicyFactoryTest, HdfsNNFailoverTest, DocExpirationUpdateProcessorFactoryTest, TestFastOutputStream, TestUnifiedSolrHighlighter, ConcurrentDeleteAndCreateCollectionTest, CdcrReplicationDistributedZkTest, ZkFailoverTest, QueryResultKeyTest, QueryParsingTest, TestFoldingMultitermQuery, TestTolerantUpdateProcessorRandomCloud, TestSimpleTextCodec, CopyFieldTest, TestInPlaceUpdatesStandalone, DistributedSuggestComponentTest, TestCustomStream, LeaderFailoverAfterPartitionTest, TestCloudRecovery, TestFieldTypeCollectionResource, BitVectorTest, LeaderElectionContextKeyTest, ConnectionReuseTest, SolrMetricManagerTest, MissingSegmentRecoveryTest, TestCustomDocTransformer, MetricUtilsTest, AnalysisErrorHandlingTest, TestDocTermOrdsUninvertLimit]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDocTermOrdsUninvertLimit -Dtests.seed=F5FFEC26479F8168 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=Asia/Yangon -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestDocTermOrdsUninvertLimit (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=80760, name=Lucene Merge Thread #584, state=RUNNABLE, group=TGRP-TestDocTermOrdsUninvertLimit]
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot complete merge
[junit4] > at __randomizedtesting.SeedInfo.seed([F5FFEC26479F8168]:0)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
[junit4] > Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot complete merge
[junit4] > at org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:3882)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4566)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4059)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:424)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:293)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:373)
[junit4] > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:113)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:82)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:172)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:211)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105)
[junit4] > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:523)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:398)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:332)
[junit4] > at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:67)
[junit4] Completed [762/820 (5!)] on J0 in 80.65s, 1 test, 2 errors <<< FAILURES!
[...truncated 192 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180630_160925_206649658939755341913.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/heapdumps/java_pid20229.hprof ...
[junit4] Heap dump file created [584509268 bytes in 3.436 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 9429 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:661: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid20229.hprof
Total time: 515 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 16 -
Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/16/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestPullReplica
Error Message:
ObjectTracker found 3 object(s) that were not released!!! [SolrCore, RawDirectoryWrapper, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1043) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:867) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:94) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:380) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [SolrCore, RawDirectoryWrapper, MockDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1043)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:867)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1140)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1050)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:94)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:380)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:504)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:347)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([ED1B0E4F1ACD66D7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:304)
at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 1) Thread[id=2584, name=searcherExecutor-983-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2793, name=searcherExecutor-1045-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2831, name=searcherExecutor-1059-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=2628, name=searcherExecutor-997-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2710, name=searcherExecutor-1025-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=2867, name=searcherExecutor-1073-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2666, name=searcherExecutor-1011-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:
1) Thread[id=2584, name=searcherExecutor-983-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2793, name=searcherExecutor-1045-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2831, name=searcherExecutor-1059-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=2628, name=searcherExecutor-997-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=2710, name=searcherExecutor-1025-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=2867, name=searcherExecutor-1073-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=2666, name=searcherExecutor-1011-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([ED1B0E4F1ACD66D7]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2584, name=searcherExecutor-983-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2793, name=searcherExecutor-1045-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2831, name=searcherExecutor-1059-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=2628, name=searcherExecutor-997-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2710, name=searcherExecutor-1025-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=2867, name=searcherExecutor-1073-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2666, name=searcherExecutor-1011-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2584, name=searcherExecutor-983-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2793, name=searcherExecutor-1045-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2831, name=searcherExecutor-1059-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=2628, name=searcherExecutor-997-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=2710, name=searcherExecutor-1025-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=2867, name=searcherExecutor-1073-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=2666, name=searcherExecutor-1011-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([ED1B0E4F1ACD66D7]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
shard1 is not consistent. Got 2310 from https://127.0.0.1:60678/collection1_shard1_replica_n21 (previous client) and got 753 from https://127.0.0.1:39817/collection1_shard1_replica_n23
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 2310 from https://127.0.0.1:60678/collection1_shard1_replica_n21 (previous client) and got 753 from https://127.0.0.1:39817/collection1_shard1_replica_n23
at __randomizedtesting.SeedInfo.seed([ED1B0E4F1ACD66D7:654F3195B4310B2F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1330)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1309)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13400 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
[junit4] 2> 403348 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/init-core-data-001
[junit4] 2> 403349 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 403349 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 403351 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 403351 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 405039 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 406188 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 406492 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Logging to org.apache.logging.slf4j.Log4jLogger@55dfd618 via org.mortbay.log.Slf4jLog
[junit4] 2> 406525 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 407007 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 407061 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58384_hdfs____z5eydy/webapp
[junit4] 2> 409944 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58384
[junit4] 2> 411805 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 411811 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 411826 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41458_datanode____nsu61t/webapp
[junit4] 2> 412424 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41458
[junit4] 2> 413993 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 413996 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 414052 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58815_datanode____.793qv4/webapp
[junit4] 2> 414845 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[ED1B0E4F1ACD66D7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58815
[junit4] 2> 417240 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:43983) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 417273 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:43983) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 417782 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6adbbe9a4c414c: from storage DS-63509ce0-8318-4b2b-abaf-af23f5a2c684 node DatanodeRegistration(127.0.0.1:58025, datanodeUuid=198c5bc1-ad7c-4c32-9af3-0450388c29e4, infoPort=42000, infoSecurePort=0, ipcPort=59734, storageInfo=lv=-56;cid=testClusterID;nsid=1768232477;c=0), blocks: 0, hasStaleStorage: true, processing time: 4 msecs
[junit4] 2> 417791 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6adbbe9a4c414c: from storage DS-79db3a76-e417-4482-92f7-148edb348582 node DatanodeRegistration(127.0.0.1:58351, datanodeUuid=d6a13c62-25fc-49b4-872c-e1b91b20bced, infoPort=45833, infoSecurePort=0, ipcPort=40113, storageInfo=lv=-56;cid=testClusterID;nsid=1768232477;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 417791 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6adbbe9a4c414c: from storage DS-cb221a91-9678-424d-b3b7-92d535003103 node DatanodeRegistration(127.0.0.1:58025, datanodeUuid=198c5bc1-ad7c-4c32-9af3-0450388c29e4, infoPort=42000, infoSecurePort=0, ipcPort=59734, storageInfo=lv=-56;cid=testClusterID;nsid=1768232477;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 417791 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x6adbbe9a4c414c: from storage DS-4789e804-1e71-4a39-b1ce-667048e7f314 node DatanodeRegistration(127.0.0.1:58351, datanodeUuid=d6a13c62-25fc-49b4-872c-e1b91b20bced, infoPort=45833, infoSecurePort=0, ipcPort=40113, storageInfo=lv=-56;cid=testClusterID;nsid=1768232477;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 418524 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 418539 INFO (Thread-572) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 418539 INFO (Thread-572) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 418545 ERROR (Thread-572) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 418690 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.ZkTestServer start zk server on port:55816
[junit4] 2> 418704 INFO (zkConnectionManagerCallback-699-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 418715 INFO (zkConnectionManagerCallback-701-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 418725 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 418731 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 418733 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 418734 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 418736 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 418737 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 418739 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 418740 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 418742 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 418743 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 418745 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 418752 INFO (zkConnectionManagerCallback-705-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 418754 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 418917 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 418924 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 418924 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 418924 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 418924 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@180ad447{/,null,AVAILABLE}
[junit4] 2> 418926 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e0fa923{SSL,[ssl, http/1.1]}{127.0.0.1:49359}
[junit4] 2> 418926 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.Server Started @419041ms
[junit4] 2> 418926 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:43983/hdfs__localhost_43983__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-BadApples-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001_tempDir-002_control_data, hostContext=/, hostPort=49359, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/control-001/cores}
[junit4] 2> 418927 ERROR (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 418927 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 418927 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 418927 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 418927 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 418927 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-24T07:41:03.423Z
[junit4] 2> 418938 INFO (zkConnectionManagerCallback-707-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 418940 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 418940 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/control-001/solr.xml
[junit4] 2> 418960 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 418960 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 418966 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 419229 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55816/solr
[junit4] 2> 419241 INFO (zkConnectionManagerCallback-711-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 419287 INFO (zkConnectionManagerCallback-713-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 419440 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 419441 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49359_
[junit4] 2> 419442 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.Overseer Overseer (id=74028783229796357-127.0.0.1:49359_-n_0000000000) starting
[junit4] 2> 419618 INFO (zkConnectionManagerCallback-720-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 419621 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55816/solr ready
[junit4] 2> 419643 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49359_
[junit4] 2> 419655 INFO (OverseerStateUpdate-74028783229796357-127.0.0.1:49359_-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 419687 INFO (zkCallback-719-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 419763 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 419867 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 419918 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 419918 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 419920 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:49359_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/control-001/cores
[junit4] 2> 420118 INFO (zkConnectionManagerCallback-726-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 420119 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 420120 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55816/solr ready
[junit4] 2> 420156 INFO (qtp150297686-2543) [n:127.0.0.1:49359_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:49359_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 420177 INFO (OverseerThreadFactory-978-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 420305 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 420305 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 421360 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 421426 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 421614 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 421653 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 421654 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 421654 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 421654 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 421658 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=27.3046875, floorSegmentMB=1.7529296875, forceMergeDeletesPctAllowed=23.017685617524307, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6521495338112513, reclaimDeletesWeight=1.4306689689244192
[junit4] 2> 421679 WARN (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 421815 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 421815 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 421915 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 30000ms;
[junit4] 2> 421915 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 3000ms;
[junit4] 2> 421919 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.14487363382645074]
[junit4] 2> 421919 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@76d53a05[control_collection_shard1_replica_n1] main]
[junit4] 2> 421921 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 421922 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 421923 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 421925 INFO (searcherExecutor-983-thread-1-processing-n:127.0.0.1:49359_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@76d53a05[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 421932 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604138897430806528
[junit4] 2> 421939 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 421943 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 421943 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 421943 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:49359/control_collection_shard1_replica_n1/
[junit4] 2> 421943 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 421944 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:49359/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 421944 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 421948 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:49359/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 422055 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 422058 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1753
[junit4] 2> 422064 INFO (qtp150297686-2543) [n:127.0.0.1:49359_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 422158 INFO (zkCallback-712-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 422178 INFO (OverseerCollectionConfigSetProcessor-74028783229796357-127.0.0.1:49359_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 423065 INFO (qtp150297686-2543) [n:127.0.0.1:49359_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:49359_&wt=javabin&version=2} status=0 QTime=2908
[junit4] 2> 423084 INFO (zkConnectionManagerCallback-731-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 423085 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 423086 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55816/solr ready
[junit4] 2> 423086 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 423090 INFO (qtp150297686-2539) [n:127.0.0.1:49359_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 423095 INFO (OverseerThreadFactory-978-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 423097 WARN (OverseerThreadFactory-978-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 423312 INFO (qtp150297686-2539) [n:127.0.0.1:49359_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 423312 INFO (qtp150297686-2539) [n:127.0.0.1:49359_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=221
[junit4] 2> 423479 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001 of type NRT
[junit4] 2> 423485 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 423488 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 423488 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 423488 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 423488 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@232d8ac4{/,null,AVAILABLE}
[junit4] 2> 423489 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@26fa6e5e{SSL,[ssl, http/1.1]}{127.0.0.1:60678}
[junit4] 2> 423489 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.e.j.s.Server Started @423604ms
[junit4] 2> 423489 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:43983/hdfs__localhost_43983__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-BadApples-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=60678, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001/cores}
[junit4] 2> 423489 ERROR (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 423489 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 423489 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 423490 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 423490 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 423490 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-24T07:41:07.986Z
[junit4] 2> 423493 INFO (zkConnectionManagerCallback-733-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 423495 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 423495 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001/solr.xml
[junit4] 2> 423499 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 423499 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 423501 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 423594 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55816/solr
[junit4] 2> 423610 INFO (zkConnectionManagerCallback-737-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 423613 INFO (zkConnectionManagerCallback-739-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 423624 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 423627 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 423630 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 423630 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60678_
[junit4] 2> 423631 INFO (zkCallback-730-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 423631 INFO (zkCallback-719-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 423632 INFO (zkCallback-712-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 423641 INFO (zkCallback-738-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 423660 INFO (zkConnectionManagerCallback-746-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 423661 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 423662 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55816/solr ready
[junit4] 2> 423663 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 423691 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 423706 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 423706 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 423709 INFO (TEST-HdfsRestartWhileUpdatingTest.test-seed#[ED1B0E4F1ACD66D7]) [n:127.0.0.1:60678_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001/cores
[junit4] 2> 423774 INFO (qtp150297686-2542) [n:127.0.0.1:49359_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60678_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 423777 INFO (OverseerCollectionConfigSetProcessor-74028783229796357-127.0.0.1:49359_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 423778 INFO (OverseerThreadFactory-978-thread-3) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:60678_ for creating new replica
[junit4] 2> 423804 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 424822 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 424836 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 424936 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 424960 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 424960 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de90129
[junit4] 2> 424961 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 424961 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_ED1B0E4F1ACD66D7-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 424965 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=27.3046875, floorSegmentMB=1.7529296875, forceMergeDeletesPctAllowed=23.017685617524307, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6521495338112513, reclaimDeletesWeight=1.4306689689244192
[junit4] 2> 424978 WARN (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 425046 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 425046 INFO (qtp348024986-2595) [n:127.0.0.1:60678_ c:collection1 s:shard1 x:collection1_shard1_r
[...truncated too long message...]
op-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null151383609
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null151383609
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null151383609
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null544704337
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null544704337
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null544704337
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 173ms :: artifacts dl 13ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 472 minutes 7 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-NightlyTests-master - Build # 15 -
Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-master/15/
16 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [ZkCollectionTerms] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkCollectionTerms at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkCollectionTerms.<init>(ZkCollectionTerms.java:39) at org.apache.solr.cloud.ZkController.getCollectionTerms(ZkController.java:1505) at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1524) at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1518) at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1521) at org.apache.solr.cloud.ZkController$UnloadCoreOnDeletedWatcher.onStateChanged(ZkController.java:2746) at org.apache.solr.common.cloud.ZkStateReader$Notification.run(ZkStateReader.java:1642) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [ZkCollectionTerms]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkCollectionTerms
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.cloud.ZkCollectionTerms.<init>(ZkCollectionTerms.java:39)
at org.apache.solr.cloud.ZkController.getCollectionTerms(ZkController.java:1505)
at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1524)
at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1518)
at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1521)
at org.apache.solr.cloud.ZkController$UnloadCoreOnDeletedWatcher.onStateChanged(ZkController.java:2746)
at org.apache.solr.common.cloud.ZkStateReader$Notification.run(ZkStateReader.java:1642)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([D46D7B227A42CCF4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:304)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ShardRoutingTest.test
Error Message:
Error from server at https://127.0.0.1:34413/b_: At least one of the node(s) specified [127.0.0.1:43342_b_] are not currently active in [], no action taken.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34413/b_: At least one of the node(s) specified [127.0.0.1:43342_b_] are not currently active in [], no action taken.
at __randomizedtesting.SeedInfo.seed([D46D7B227A42CCF4:5C3944F8D4BEA10C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
50 threads leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest: 1) Thread[id=9087, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 2) Thread[id=9046, name=zkCallback-1836-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=9061, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=9068, name=qtp2065354281-9068, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=9091, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=9024, name=qtp2068255422-9024, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=9067, name=qtp2065354281-9067, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=9044, name=zkConnectionManagerCallback-1837-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=9056, name=Scheduler-1555861337, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=9048, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 11) Thread[id=9064, name=qtp2065354281-9064-acceptor-0@3ed6ba11-ServerConnector@3cb04368{SSL,[ssl, http/1.1]}{127.0.0.1:43342}, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=9083, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=9097, name=Thread-2797, state=WAITING, group=TGRP-ShardRoutingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 14) Thread[id=9047, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=9117, name=SolrRrdBackendFactory-2219-thread-2, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=9043, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 17) Thread[id=9095, name=SolrRrdBackendFactory-2233-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=9118, name=SolrRrdBackendFactory-2233-thread-2, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=9051, name=SolrRrdBackendFactory-2219-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=9063, name=qtp2065354281-9063, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=9081, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=9028, name=qtp2068255422-9028, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=9032, name=Scheduler-1496323281, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=9092, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 25) Thread[id=9037, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=9082, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=9062, name=qtp2065354281-9062, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=9094, name=zkConnectionManagerCallback-1860-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=9065, name=qtp2065354281-9065, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=9038, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=9069, name=qtp2065354281-9069, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=9026, name=qtp2068255422-9026-acceptor-0@74e72a32-ServerConnector@40cec40d{SSL,[ssl, http/1.1]}{127.0.0.1:45600}, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=9049, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 34) Thread[id=9050, name=zkConnectionManagerCallback-1844-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=9058, name=searcherExecutor-2222-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=9066, name=qtp2065354281-9066, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 37) Thread[id=9030, name=qtp2068255422-9030, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=9025, name=qtp2068255422-9025, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=9027, name=qtp2068255422-9027, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 40) Thread[id=9070, name=Scheduler-357915888, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 41) Thread[id=9042, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 42) Thread[id=9029, name=qtp2068255422-9029, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 43) Thread[id=9053, name=Thread-2788, state=WAITING, group=TGRP-ShardRoutingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 44) Thread[id=9088, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 45) Thread[id=9031, name=qtp2068255422-9031, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 46) Thread[id=9052, name=MetricsHistoryHandler-2220-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 47) Thread[id=9036, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 48) Thread[id=9089, name=zkConnectionManagerCallback-1853-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 49) Thread[id=9093, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 50) Thread[id=9096, name=MetricsHistoryHandler-2234-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 50 threads leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest:
1) Thread[id=9087, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
2) Thread[id=9046, name=zkCallback-1836-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=9061, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=9068, name=qtp2065354281-9068, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=9091, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=9024, name=qtp2068255422-9024, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=9067, name=qtp2065354281-9067, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=9044, name=zkConnectionManagerCallback-1837-thread-1, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=9056, name=Scheduler-1555861337, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=9048, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
11) Thread[id=9064, name=qtp2065354281-9064-acceptor-0@3ed6ba11-ServerConnector@3cb04368{SSL,[ssl, http/1.1]}{127.0.0.1:43342}, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=9083, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=9097, name=Thread-2797, state=WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
14) Thread[id=9047, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=9117, name=SolrRrdBackendFactory-2219-thread-2, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=9043, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
17) Thread[id=9095, name=SolrRrdBackendFactory-2233-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=9118, name=SolrRrdBackendFactory-2233-thread-2, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19) Thread[id=9051, name=SolrRrdBackendFactory-2219-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20) Thread[id=9063, name=qtp2065354281-9063, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
21) Thread[id=9081, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
22) Thread[id=9028, name=qtp2068255422-9028, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
23) Thread[id=9032, name=Scheduler-1496323281, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
24) Thread[id=9092, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
25) Thread[id=9037, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
26) Thread[id=9082, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
27) Thread[id=9062, name=qtp2065354281-9062, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
28) Thread[id=9094, name=zkConnectionManagerCallback-1860-thread-1, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
29) Thread[id=9065, name=qtp2065354281-9065, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
30) Thread[id=9038, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
31) Thread[id=9069, name=qtp2065354281-9069, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
32) Thread[id=9026, name=qtp2068255422-9026-acceptor-0@74e72a32-ServerConnector@40cec40d{SSL,[ssl, http/1.1]}{127.0.0.1:45600}, state=RUNNABLE, group=TGRP-ShardRoutingTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
33) Thread[id=9049, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
34) Thread[id=9050, name=zkConnectionManagerCallback-1844-thread-1, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
35) Thread[id=9058, name=searcherExecutor-2222-thread-1, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
36) Thread[id=9066, name=qtp2065354281-9066, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
37) Thread[id=9030, name=qtp2068255422-9030, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
38) Thread[id=9025, name=qtp2068255422-9025, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
39) Thread[id=9027, name=qtp2068255422-9027, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
40) Thread[id=9070, name=Scheduler-357915888, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
41) Thread[id=9042, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
42) Thread[id=9029, name=qtp2068255422-9029, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
43) Thread[id=9053, name=Thread-2788, state=WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
44) Thread[id=9088, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
45) Thread[id=9031, name=qtp2068255422-9031, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
46) Thread[id=9052, name=MetricsHistoryHandler-2220-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
47) Thread[id=9036, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
48) Thread[id=9089, name=zkConnectionManagerCallback-1853-thread-1, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
49) Thread[id=9093, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-EventThread, state=WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
50) Thread[id=9096, name=MetricsHistoryHandler-2234-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([D46D7B227A42CCF4]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=9087, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 2) Thread[id=9046, name=zkCallback-1836-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=9024, name=qtp2068255422-9024, state=RUNNABLE, group=TGRP-ShardRoutingTest] at sun.nio.ch.IOUtil.drain(Native Method) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$30/641393836.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=9070, name=Scheduler-357915888, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=9042, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 6) Thread[id=9044, name=zkConnectionManagerCallback-1837-thread-1, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=9056, name=Scheduler-1555861337, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=9048, name=TEST-ShardRoutingTest.test-seed#[D46D7B227A42CCF4]-SendThread(127.0.0.1:33304), state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 9) Thread[id=9097, name=Thread-2797, state=WAITING, group=TGRP-ShardRoutingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 10) Thread[id=9053, name=Thread-2788, state=WAITING, group=TGRP-ShardRoutingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832) 11) Thread[id=9117, name=SolrRrdBackendFactory-2219-thread-2, state=WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=9031, name=qtp2068255422-9031, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=9095, name=SolrRrdBackendFactory-2233-thread-1, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.p
[...truncated too long message...]
y-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null966251133
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null966251133
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/null966251133
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null196610930
[copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null196610930
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/solr/null196610930
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 72ms :: artifacts dl 12ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 630 minutes 13 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)