You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/02/06 21:12:06 UTC
[JENKINS] Lucene-Solr-Tests-7.x - Build # 357 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/357/
6 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.facet.ValueFacetTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.ValueFacetTest: 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.ValueFacetTest:
1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FE64A6BBC0A09CF2]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.facet.ValueFacetTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FE64A6BBC0A09CF2]:0)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([D6E1B8D140A8A33C:5EDC31AE7A684291]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState(TriggerIntegrationTest.java:426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
Stack Trace:
java.lang.ClassCastException: org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
at __randomizedtesting.SeedInfo.seed([D6E1B8D140A8A33C:1F54FA7F49CF65C9]:0)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([D6E1B8D140A8A33C:F08B06E85BA70C50]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger(TriggerIntegrationTest.java:455)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([D6E1B8D140A8A33C:8BA9A6588F6E0573]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11782 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> 112398 INFO (SUITE-TestLargeCluster-seed#[D6E1B8D140A8A33C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_D6E1B8D140A8A33C-001/init-core-data-001
[junit4] 2> 112399 INFO (SUITE-TestLargeCluster-seed#[D6E1B8D140A8A33C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 112417 INFO (SUITE-TestLargeCluster-seed#[D6E1B8D140A8A33C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 112501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 112502 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 112502 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr, 127.0.0.1:10007_solr, 127.0.0.1:10015_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10005_solr, 127.0.0.1:10013_solr, 127.0.0.1:10021_solr, 127.0.0.1:10035_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10008_solr, 127.0.0.1:10019_solr, 127.0.0.1:10024_solr, 127.0.0.1:10027_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10025_solr, 127.0.0.1:10017_solr, 127.0.0.1:10002_solr, 127.0.0.1:10028_solr, 127.0.0.1:10036_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10006_solr, 127.0.0.1:10037_solr, 127.0.0.1:10009_solr, 127.0.0.1:10029_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10003_solr, 127.0.0.1:10011_solr]
[junit4] 2> 112538 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 112538 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 112554 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 112554 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 112577 DEBUG (ScheduledTrigger-79-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112617 INFO (TEST-TestLargeCluster.testNodeLost-seed#[D6E1B8D140A8A33C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
[junit4] 2> 112617 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[D6E1B8D140A8A33C]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 112617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 112618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 112618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 112618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 112619 INFO (TEST-TestLargeCluster.testNodeLost-seed#[D6E1B8D140A8A33C]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 112632 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 112632 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[D6E1B8D140A8A33C]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 112633 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 112634 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 112634 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 112634 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 112634 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 112647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 112647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 112649 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112680 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112712 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112733 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112757 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112777 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112798 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112822 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112842 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112879 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112897 DEBUG (simCloudManagerPool-81-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 112897 DEBUG (simCloudManagerPool-81-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 112940 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112948 DEBUG (simCloudManagerPool-81-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 112948 DEBUG (simCloudManagerPool-81-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 112949 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 112949 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 112949 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 112949 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 112968 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 112970 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113004 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113005 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113041 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113041 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113078 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113078 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113118 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113118 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113138 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113139 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113183 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113183 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113219 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113219 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113256 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113256 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113292 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113293 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113329 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113329 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113349 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113381 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113402 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113438 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113475 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113584 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113593 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113621 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113621 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113641 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113642 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113662 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113662 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113698 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113698 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113734 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113735 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113767 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113767 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113787 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113788 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113815 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113816 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113852 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113852 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113888 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113889 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113925 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113925 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113945 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113946 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 113974 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 113974 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114009 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114009 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114029 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114029 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114063 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114063 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114087 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114087 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114120 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114120 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114140 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114140 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114172 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114173 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114193 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114193 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114213 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114213 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114250 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114250 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114286 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114286 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114323 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114323 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114355 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114355 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114376 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114376 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114412 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114412 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114449 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114449 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114481 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114481 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114509 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114510 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114546 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114546 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114595 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114595 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114631 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114631 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114668 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114668 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114704 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114704 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114741 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114741 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114777 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114778 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114817 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114817 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114851 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114851 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114887 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114887 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114924 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114924 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114960 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114961 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 114997 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 114997 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115034 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115034 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115070 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115070 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115107 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115107 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115144 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115144 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115181 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115181 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115217 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115217 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115253 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115253 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115290 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115290 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115326 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115326 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115359 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115359 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115379 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115379 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115416 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115416 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115440 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115440 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115464 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115465 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115493 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115493 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115514 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115514 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115554 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115554 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115586 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115587 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115607 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115607 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115628 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115629 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115664 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115664 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115688 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115688 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115724 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115725 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115761 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115761 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115781 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115782 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115802 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115802 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115822 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115822 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115854 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115854 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115875 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115875 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115911 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115911 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115931 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115931 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115952 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115952 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 115980 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 115980 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116000 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116001 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116021 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116021 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116053 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116053 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116073 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116073 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116102 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116102 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116122 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116122 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116159 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116159 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116195 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116195 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116216 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116216 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116236 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116236 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116256 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116256 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116284 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116284 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116317 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116317 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116337 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116337 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116357 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116358 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116378 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116378 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116406 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116406 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116435 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116435 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116455 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116455 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116475 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116476 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116496 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116496 DEBUG (ScheduledTrigger-82-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116520 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116520 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116540 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116540 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116561 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116561 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116581 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116581 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116601 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116601 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116622 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116622 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116642 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116642 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116662 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116662 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116682 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116683 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116703 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116703 DEBUG (ScheduledTrigger-82-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116735 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116736 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116837 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116837 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116844 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard2: {"core_node18":{
[junit4] 2> "node_name":"127.0.0.1:10018_solr",
[junit4] 2> "core":"testNodeLost_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard13: {"core_node129":{
[junit4] 2> "node_name":"127.0.0.1:10082_solr",
[junit4] 2> "core":"testNodeLost_shard13_replica_n129",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard13",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard14: {"core_node139":{
[junit4] 2> "node_name":"127.0.0.1:10060_solr",
[junit4] 2> "core":"testNodeLost_shard14_replica_n139",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard14",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard15: {"core_node149":{
[junit4] 2> "node_name":"127.0.0.1:10054_solr",
[junit4] 2> "core":"testNodeLost_shard15_replica_n149",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard15",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard4: {"core_node39":{
[junit4] 2> "node_name":"127.0.0.1:10060_solr",
[junit4] 2> "core":"testNodeLost_shard4_replica_n39",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard16: {"core_node159":{
[junit4] 2> "node_name":"127.0.0.1:10071_solr",
[junit4] 2> "core":"testNodeLost_shard16_replica_n159",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard16",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard5: {"core_node50":{
[junit4] 2> "node_name":"127.0.0.1:10096_solr",
[junit4] 2> "core":"testNodeLost_shard5_replica_n50",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard17: {"core_node169":{
[junit4] 2> "node_name":"127.0.0.1:10065_solr",
[junit4] 2> "core":"testNodeLost_shard17_replica_n169",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard17",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard18: {"core_node180":{
[junit4] 2> "node_name":"127.0.0.1:10033_solr",
[junit4] 2> "core":"testNodeLost_shard18_replica_n180",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard18",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116845 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard19: {"core_node190":{
[junit4] 2> "node_name":"127.0.0.1:10025_solr",
[junit4] 2> "core":"testNodeLost_shard19_replica_n190",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard19",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116846 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard1: {"core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10021_solr",
[junit4] 2> "core":"testNodeLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116846 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard6: {"core_node60":{
[junit4] 2> "node_name":"127.0.0.1:10094_solr",
[junit4] 2> "core":"testNodeLost_shard6_replica_n60",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard6",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116846 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard7: {"core_node70":{
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "core":"testNodeLost_shard7_replica_n70",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard7",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116846 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard8: {"core_node80":{
[junit4] 2> "node_name":"127.0.0.1:10033_solr",
[junit4] 2> "core":"testNodeLost_shard8_replica_n80",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard8",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116846 DEBUG (simCloudManagerPool-81-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard9: {"core_node90":{
[junit4] 2> "node_name":"127.0.0.1:10025_solr",
[junit4] 2> "core":"testNodeLost_shard9_replica_n90",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard9",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116848 DEBUG (simCloudManagerPool-81-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard11: {"core_node108":{
[junit4] 2> "node_name":"127.0.0.1:10048_solr",
[junit4] 2> "core":"testNodeLost_shard11_replica_n108",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard11",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116848 DEBUG (simCloudManagerPool-81-thread-19) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard12: {"core_node119":{
[junit4] 2> "node_name":"127.0.0.1:10035_solr",
[junit4] 2> "core":"testNodeLost_shard12_replica_n119",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard12",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116848 DEBUG (simCloudManagerPool-81-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard20: {"core_node191":{
[junit4] 2> "node_name":"127.0.0.1:10013_solr",
[junit4] 2> "core":"testNodeLost_shard20_replica_n191",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard20",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116848 DEBUG (simCloudManagerPool-81-thread-37) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard3: {"core_node29":{
[junit4] 2> "node_name":"127.0.0.1:10082_solr",
[junit4] 2> "core":"testNodeLost_shard3_replica_n29",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116848 DEBUG (simCloudManagerPool-81-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard10: {"core_node92":{
[junit4] 2> "node_name":"127.0.0.1:10079_solr",
[junit4] 2> "core":"testNodeLost_shard10_replica_n92",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard10",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 116873 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116873 DEBUG (ScheduledTrigger-82-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 116902 DEBUG (ScheduledTrigger-82-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 116902 DEBUG (ScheduledTri
[...truncated too long message...]
serPlugin, TestSchemaNameResource, TestHdfsUpdateLog, TermsComponentTest, PKIAuthenticationIntegrationTest, ChaosMonkeySafeLeaderTest, TestJettySolrRunner, RulesTest, TestExportWriter, CdcrReplicationDistributedZkTest, DistributedFacetSimpleRefinementLongTailTest, LegacyCloudClusterPropTest, TestQueryUtils, HLLSerializationTest, IndexSchemaTest, SystemInfoHandlerTest, RestartWhileUpdatingTest, SortSpecParsingTest, HighlighterMaxOffsetTest, TestOnReconnectListenerSupport, ZkStateWriterTest, TestSolr4Spatial, BlockCacheTest, DocExpirationUpdateProcessorFactoryTest, OverriddenZkACLAndCredentialsProvidersTest, TestSchemaVersionResource, SolrGangliaReporterTest, TestZkChroot, OverseerRolesTest, TestJavabinTupleStreamParser, TestSystemIdResolver, DeleteStatusTest, TestCharFilters, HighlighterTest, TestCollationFieldDocValues, TestSchemaManager, InfixSuggestersTest, TestReplicationHandler, TestLRUCache, CollectionTooManyReplicasTest, ScriptEngineTest, SuggesterTest, TestFieldResource, HttpPartitionTest, XsltUpdateRequestHandlerTest, SimplePostToolTest, TestCoreBackup, SampleTest, TestMaxScoreQueryParser, AtomicUpdateProcessorFactoryTest, BasicFunctionalityTest, TestClassicSimilarityFactory, TestImpersonationWithHadoopAuth, DistributedMLTComponentTest, TestRestManager, PolyFieldTest, DistributedDebugComponentTest, TestDynamicFieldCollectionResource, BitVectorTest, ShardRoutingCustomTest, RequiredFieldsTest, TestCloudDeleteByQuery, QueryParsingTest, OverseerTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, TestHashPartitioner, TestRealTimeGet, DistributedTermsComponentTest, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, StatsComponentTest, BadIndexSchemaTest, TestSort, TestLazyCores, TestFoldingMultitermQuery, DocValuesTest, MoreLikeThisHandlerTest, FastVectorHighlighterTest, RegexBoostProcessorTest, ReturnFieldsTest, JsonLoaderTest, TestPHPSerializedResponseWriter, SearchHandlerTest, SOLR749Test, TestQuerySenderListener, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestBinaryField, TestSearchPerf, MinimalSchemaTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestCodecSupport, SynonymTokenizerTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, PluginInfoTest, DOMUtilTest, RAMDirectoryFactoryTest, TestSolrJ, ZkNodePropsTest, SliceStateTest, FileUtilsTest, CircularListTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestLuceneIndexBackCompat, AliasIntegrationTest, CleanupOldIndexTest, ConfigSetsAPITest, CreateRoutedAliasTest, DeleteInactiveReplicaTest, DeleteShardTest, DistribCursorPagingTest, LeaderElectionContextKeyTest, MigrateRouteKeyTest, MoveReplicaHDFSTest, RecoveryAfterSoftCommitTest, TestCloudRecovery, TestLeaderInitiatedRecoveryThread, TestRebalanceLeaders, TestSegmentSorting, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation, TestTlogReplica, MetricTriggerTest, SearchRateTriggerTest, TriggerIntegrationTest]
[junit4] Completed [651/774 (2!)] on J1 in 235.51s, 14 tests, 2 failures, 1 error <<< FAILURES!
[...truncated 3603 lines...]
[junit4] Suite: org.apache.solr.analytics.facet.ValueFacetTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analytics/test/J0/temp/solr.analytics.facet.ValueFacetTest_FE64A6BBC0A09CF2-001/init-core-data-001
[junit4] 2> Feb 06, 2018 8:46:00 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> Feb 06, 2018 8:46:20 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.ValueFacetTest:
[junit4] 2> 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 06, 2018 8:46:20 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
[junit4] 2> Feb 06, 2018 8:46:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=4, maxDocsPerChunk=397, blockSize=378), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=4, blockSize=378)), sim=RandomSimilarity(queryNorm=true): {}, locale=uk-UA, timezone=Canada/Newfoundland
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=195373336,total=307232768
[junit4] 2> NOTE: All tests run in this JVM: [LegacyNoFacetTest, BottomFunctionTest, FilterFunctionTest, EqualFunctionTest, CastingStringValueStreamTest, ValueFacetTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ValueFacetTest -Dtests.seed=FE64A6BBC0A09CF2 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=uk-UA -Dtests.timezone=Canada/Newfoundland -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ValueFacetTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.ValueFacetTest:
[junit4] > 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([FE64A6BBC0A09CF2]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=72, name=qtp250559835-72, state=TIMED_WAITING, group=TGRP-ValueFacetTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([FE64A6BBC0A09CF2]:0)
[junit4] Completed [52/78 (1!)] on J0 in 71.80s, 11 tests, 2 errors <<< FAILURES!
[...truncated 45074 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 375 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/375/
2 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:38040/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:36339/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:38040/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:36339/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([1C5A1C03E5256FF7:B697CFF152F6BA27]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:991)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:306)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36339/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 46 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud.testCreateCollectionAddShardUsingPolicy
Error Message:
Timeout waiting for collection to become active Live Nodes: null Last available state: null
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: null
Last available state: null
at __randomizedtesting.SeedInfo.seed([1C5A1C03E5256FF7:DDC62EAA241D9234]:0)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:230)
at org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud.testCreateCollectionAddShardUsingPolicy(TestPolicyCloud.java:305)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.lambda$null$52(SimClusterStateProvider.java:1226)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.lambda$getCollectionStates$53(SimClusterStateProvider.java:1218)
at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getCollectionStates(SimClusterStateProvider.java:1217)
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getClusterState(SimClusterStateProvider.java:1199)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:254)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:224)
... 41 more
Build Log:
[...truncated 12414 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud
[junit4] 2> 250054 INFO (SUITE-TestPolicyCloud-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestPolicyCloud_1C5A1C03E5256FF7-001/init-core-data-001
[junit4] 2> 250054 WARN (SUITE-TestPolicyCloud-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 250054 INFO (SUITE-TestPolicyCloud-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 250055 INFO (SUITE-TestPolicyCloud-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 250073 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddReplica
[junit4] 2> 250082 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 250091 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 250258 INFO (simCloudManagerPool-1611-thread-2) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676385026 sessionWrapper.createTime 18676384067489154, this.sessionWrapper.createTime 18676384067489154
[junit4] 2> 250315 INFO (simCloudManagerPool-1611-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676387878 sessionWrapper.createTime 18676387533852254, this.sessionWrapper.createTime 18676387533852254
[junit4] 2> 250315 INFO (simCloudManagerPool-1611-thread-4) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
[junit4] 2> 250337 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddReplica
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 4
[junit4] 2> 250338 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 250339 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testCreateCollectionAddReplica]
[junit4] 2> 250339 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 2
[junit4] 2> 250339 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 250339 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 2
[junit4] 2> 250340 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * testCreateCollectionAddReplica 2
[junit4] 2> 250340 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 2
[junit4] 2> 250340 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 250341 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - ADDREPLICA 1
[junit4] 2> 250341 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 250341 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 250341 INFO (TEST-TestPolicyCloud.testCreateCollectionAddReplica-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 250346 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProvider
[junit4] 2> 250346 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 250346 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProvider
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 250511 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [policiesTest]
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 1
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 2
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * policiesTest 2
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 2
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - ADDROLE 1
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 250512 INFO (TEST-TestPolicyCloud.testDataProvider-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 250515 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricsTag
[junit4] 2> 250515 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 250515 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 250619 ERROR (simCloudManagerPool-1617-thread-1) [ ] o.a.s.h.RequestHandlerBase org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{set-cluster-policy=[{cores=<10, node=#ANY}, {replica=<2, shard=#EACH, node=#ANY}, {metrics:abc=overseer, replica=0}], errorMessages=[Invalid metrics: param in {
[junit4] 2> "metrics:abc":"overseer",
[junit4] 2> "replica":0} must have at 2 or 3 segments after 'metrics:' separated by ':']}],
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.processOps(AutoScalingHandler.java:207)
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleRequestBody(AutoScalingHandler.java:145)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:497)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$2(SimCloudManager.java:459)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 250642 INFO (simCloudManagerPool-1617-thread-3) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676404250 sessionWrapper.createTime 18676404048761504, this.sessionWrapper.createTime 18676404048761504
[junit4] 2> 250661 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMetricsTag
[junit4] 2> 250661 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 250661 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 4
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [metricsTest]
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 1
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 1
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * metricsTest 1
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 1
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 250662 INFO (TEST-TestPolicyCloud.testMetricsTag-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 250665 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] 2> 250665 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 250665 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 250766 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.TestPolicyCloud NRT 127.0.0.1:10001_solr PULL 127.0.0.1:10004_solr , TLOG 127.0.0.1:10004_solr
[junit4] 2> 250791 INFO (simCloudManagerPool-1620-thread-2) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676411692 sessionWrapper.createTime 18676411541969104, this.sessionWrapper.createTime 18676411541969104
[junit4] 2> 250934 INFO (simCloudManagerPool-1620-thread-6) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676418849 sessionWrapper.createTime 18676418566500754, this.sessionWrapper.createTime 18676418566500754
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [policiesTest]
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 4
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 6
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * policiesTest 6
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 6
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATESHARD 1
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 250976 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 250980 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSplitShard
[junit4] 2> 250980 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 250980 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 251102 INFO (simCloudManagerPool-1623-thread-2) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676427257 sessionWrapper.createTime 18676426883744754, this.sessionWrapper.createTime 18676426883744754
[junit4] 2> 251270 INFO (simCloudManagerPool-1623-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676435623 sessionWrapper.createTime 18676434359083754, this.sessionWrapper.createTime 18676434359083754
[junit4] 2> 251440 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSplitShard
[junit4] 2> 251440 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 251440 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 5
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - DELETE 1
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SPLITSHARD 1
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 251441 INFO (TEST-TestPolicyCloud.testCreateCollectionSplitShard-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 251445 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProviderPerReplicaDetails
[junit4] 2> 251446 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 251446 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProviderPerReplicaDetails
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 5
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 251735 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - DELETE 1
[junit4] 2> 251736 INFO (TEST-TestPolicyCloud.testDataProviderPerReplicaDetails-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 251740 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddShardUsingPolicy
[junit4] 2> 251740 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 251740 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 251861 INFO (simCloudManagerPool-1629-thread-2) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676465210 sessionWrapper.createTime 18676465131490404, this.sessionWrapper.createTime 18676465131490404
[junit4] 2> 251988 INFO (simCloudManagerPool-1629-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18676471542 sessionWrapper.createTime 18676471517675404, this.sessionWrapper.createTime 18676471517675404
[junit4] 2> 252001 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddShardUsingPolicy
[junit4] 2> 252001 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 252001 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 252001 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 252001 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 5
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 4
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [policiesTest]
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 3
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 3
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * policiesTest 3
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 3
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATESHARD 1
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 252002 INFO (TEST-TestPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestPolicyCloud -Dtests.method=testCreateCollectionAddShardUsingPolicy -Dtests.seed=1C5A1C03E5256FF7 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-SA -Dtests.timezone=Africa/Banjul -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.27s J1 | TestPolicyCloud.testCreateCollectionAddShardUsingPolicy <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for collection to become active
[junit4] > Live Nodes: null
[junit4] > Last available state: null
[junit4] > at __randomizedtesting.SeedInfo.seed([1C5A1C03E5256FF7:DDC62EAA241D9234]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:230)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud.testCreateCollectionAddShardUsingPolicy(TestPolicyCloud.java:305)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.NullPointerException
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.lambda$null$52(SimClusterStateProvider.java:1226)
[junit4] > at java.util.ArrayList.forEach(ArrayList.java:1249)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.lambda$getCollectionStates$53(SimClusterStateProvider.java:1218)
[junit4] > at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getCollectionStates(SimClusterStateProvider.java:1217)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getClusterState(SimClusterStateProvider.java:1199)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:254)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:224)
[junit4] > ... 41 more
[junit4] 2> 252004 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestPolicyCloud_1C5A1C03E5256FF7-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1366, maxMBSortInHeap=5.290165823604289, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-SA, timezone=Africa/Banjul
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=113994840,total=433586176
[junit4] 2> NOTE: All tests run in this JVM: [TestReload, ConjunctionSolrSpellCheckerTest, TestCustomSort, TestTrie, FileUtilsTest, FullHLLTest, SpatialRPTFieldTypeTest, TestPayloadCheckQParserPlugin, DocValuesNotIndexedTest, DirectUpdateHandlerOptimizeTest, JsonLoaderTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestNRTOpen, TestExclusionRuleCollectionAccess, TestSchemaSimilarityResource, ZkControllerTest, ResponseBuilderTest, TestDelegationWithHadoopAuth, TestSubQueryTransformerCrossCore, SolrCoreMetricManagerTest, TestDocTermOrds, TestDistribStateManager, TestRTGBase, TestZkChroot, UtilsToolTest, SegmentsInfoRequestHandlerTest, AutoAddReplicasPlanActionTest, SimpleMLTQParserTest, TestLegacyNumericRangeQueryBuilder, RestartWhileUpdatingTest, DistributedFacetPivotSmallAdvancedTest, SparseHLLTest, TestSolr4Spatial, CollectionStateFormat2Test, TestCorePropertiesReload, TestRawResponseWriter, SignatureUpdateProcessorFactoryTest, ClassificationUpdateProcessorFactoryTest, CdcrReplicationDistributedZkTest, SortByFunctionTest, HdfsNNFailoverTest, TestCollectionAPIs, LoggingHandlerTest, HdfsRestartWhileUpdatingTest, DistributedFacetExistsSmallTest, TestSolrQueryResponse, DistribJoinFromCollectionTest, TestReversedWildcardFilterFactory, TestConfigSetImmutable, SyncSliceTest, CheckHdfsIndexTest, BaseCdcrDistributedZkTest, SuggesterFSTTest, TestPolicyCloud]
[junit4] Completed [102/775 (1!)] on J1 in 1.98s, 7 tests, 1 failure <<< FAILURES!
[...truncated 216 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> 524316 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/init-core-data-001
[junit4] 2> 524317 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 524317 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 524318 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 524318 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001
[junit4] 2> 524319 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 524319 INFO (Thread-1096) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 524319 INFO (Thread-1096) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 524321 ERROR (Thread-1096) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 524419 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:60863
[junit4] 2> 524421 INFO (zkConnectionManagerCallback-890-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524423 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 524430 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 524431 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 524432 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 524439 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 524439 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 524439 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 524439 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f86b472{/solr,null,AVAILABLE}
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6b736ee2{HTTP/1.1,[http/1.1]}{127.0.0.1:38040}
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.e.j.s.Server Started @526566ms
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38040}
[junit4] 2> 524441 ERROR (jetty-launcher-887-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 524441 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:39:01.480Z
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6644b5f2{/solr,null,AVAILABLE}
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61067959{HTTP/1.1,[http/1.1]}{127.0.0.1:36339}
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.e.j.s.Server Started @526569ms
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36339}
[junit4] 2> 524443 ERROR (jetty-launcher-887-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 524443 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 524444 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 524444 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 524444 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:39:01.483Z
[junit4] 2> 524449 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 524449 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 524449 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 524449 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1621a0ac{/solr,null,AVAILABLE}
[junit4] 2> 524454 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 524454 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 524454 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@de81b61{/solr,null,AVAILABLE}
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1d36b855{HTTP/1.1,[http/1.1]}{127.0.0.1:49142}
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.e.j.s.Server Started @526581ms
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49142}
[junit4] 2> 524455 ERROR (jetty-launcher-887-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 524455 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:39:01.494Z
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@53c7e5a3{HTTP/1.1,[http/1.1]}{127.0.0.1:35232}
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.e.j.s.Server Started @526581ms
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35232}
[junit4] 2> 524456 ERROR (jetty-launcher-887-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 524456 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:39:01.495Z
[junit4] 2> 524458 INFO (zkConnectionManagerCallback-894-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524459 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 524463 INFO (zkConnectionManagerCallback-892-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524463 INFO (zkConnectionManagerCallback-898-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524465 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 524465 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 524470 INFO (zkConnectionManagerCallback-896-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524471 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 524485 INFO (jetty-launcher-887-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60863/solr
[junit4] 2> 524489 INFO (jetty-launcher-887-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60863/solr
[junit4] 2> 524490 INFO (jetty-launcher-887-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60863/solr
[junit4] 2> 524493 INFO (jetty-launcher-887-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60863/solr
[junit4] 2> 524497 INFO (zkConnectionManagerCallback-907-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524497 INFO (zkConnectionManagerCallback-912-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524499 INFO (zkConnectionManagerCallback-903-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524514 INFO (zkConnectionManagerCallback-914-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524515 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10459370e090008, likely client has closed socket
[junit4] 2> 524518 INFO (zkConnectionManagerCallback-916-thread-1-processing-n:127.0.0.1:38040_solr) [n:127.0.0.1:38040_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524530 INFO (zkConnectionManagerCallback-918-thread-1-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524531 INFO (zkConnectionManagerCallback-920-thread-1-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524538 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 524539 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 524539 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35232_solr
[junit4] 2> 524540 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36339_solr
[junit4] 2> 524540 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.c.Overseer Overseer (id=73281586938314763-127.0.0.1:35232_solr-n_0000000000) starting
[junit4] 2> 524555 INFO (zkConnectionManagerCallback-922-thread-1-processing-n:127.0.0.1:49142_solr) [n:127.0.0.1:49142_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524560 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 524561 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 524565 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49142_solr
[junit4] 2> 524600 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35232_solr
[junit4] 2> 524603 INFO (zkCallback-917-thread-1-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 524603 INFO (zkCallback-919-thread-1-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 524607 INFO (OverseerStateUpdate-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 524614 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 524615 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 524616 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38040_solr
[junit4] 2> 524618 INFO (zkCallback-917-thread-2-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 524618 INFO (zkCallback-919-thread-2-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 524677 INFO (zkCallback-921-thread-1-processing-n:127.0.0.1:49142_solr) [n:127.0.0.1:49142_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 524681 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:38040_solr) [n:127.0.0.1:38040_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 524718 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49142.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524729 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36339.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524731 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49142.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524731 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49142.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524732 INFO (jetty-launcher-887-thread-4) [n:127.0.0.1:49142_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001/node4/.
[junit4] 2> 524736 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36339.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524737 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36339.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524740 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35232.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524745 INFO (jetty-launcher-887-thread-2) [n:127.0.0.1:36339_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001/node2/.
[junit4] 2> 524748 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35232.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524748 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35232.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524749 INFO (jetty-launcher-887-thread-1) [n:127.0.0.1:35232_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001/node1/.
[junit4] 2> 524822 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38040.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524840 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38040.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524840 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38040.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 524841 INFO (jetty-launcher-887-thread-3) [n:127.0.0.1:38040_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001/node3/.
[junit4] 2> 524873 INFO (zkConnectionManagerCallback-932-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524877 INFO (zkConnectionManagerCallback-936-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 524878 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 524879 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60863/solr ready
[junit4] 2> 524885 INFO (qtp691893789-4840) [n:127.0.0.1:35232_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 524888 INFO (qtp691893789-4840) [n:127.0.0.1:35232_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=3
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 524922 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 524933 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 524937 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 524945 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_47328_hdfs____8twoao/webapp
[junit4] 2> 525340 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47328
[junit4] 2> 525429 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 525429 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 525438 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40885_datanode____.km4uhs/webapp
[junit4] 2> 525850 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40885
[junit4] 2> 525936 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 525938 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 525953 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47359_datanode____edye8r/webapp
[junit4] 2> 526115 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 526146 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x425a50bebf73d0: from storage DS-a45bcbe1-e4b1-4dab-8c49-34f98afdbd0a node DatanodeRegistration(127.0.0.1:50834, datanodeUuid=102aab93-1c54-47bb-877e-04628b4564bd, infoPort=39048, infoSecurePort=0, ipcPort=56147, storageInfo=lv=-56;cid=testClusterID;nsid=1177703602;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 526146 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x425a50bebf73d0: from storage DS-75d92094-3b29-4036-b27f-bcfd1ec3a440 node DatanodeRegistration(127.0.0.1:50834, datanodeUuid=102aab93-1c54-47bb-877e-04628b4564bd, infoPort=39048, infoSecurePort=0, ipcPort=56147, storageInfo=lv=-56;cid=testClusterID;nsid=1177703602;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 526435 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47359
[junit4] 2> 526587 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 526605 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x425a50da1d7ca9: from storage DS-59733077-9ee2-4955-b230-6f538400caff node DatanodeRegistration(127.0.0.1:37554, datanodeUuid=514e0124-c7a8-47ae-a4b3-2b5144910f16, infoPort=47998, infoSecurePort=0, ipcPort=50991, storageInfo=lv=-56;cid=testClusterID;nsid=1177703602;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 526605 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x425a50da1d7ca9: from storage DS-43bc813f-e297-48e5-95a0-d0a88a04d074 node DatanodeRegistration(127.0.0.1:37554, datanodeUuid=514e0124-c7a8-47ae-a4b3-2b5144910f16, infoPort=47998, infoSecurePort=0, ipcPort=50991, storageInfo=lv=-56;cid=testClusterID;nsid=1177703602;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 526700 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testFailedMove
[junit4] 2> 526700 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 526702 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 526711 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 526711 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 526711 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 526711 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52278b1d{/solr,null,AVAILABLE}
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2532d7bb{HTTP/1.1,[http/1.1]}{127.0.0.1:54371}
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.e.j.s.Server Started @528837ms
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54371}
[junit4] 2> 526712 ERROR (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 526712 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:39:03.751Z
[junit4] 2> 526719 INFO (zkConnectionManagerCallback-939-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 526719 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 526730 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60863/solr
[junit4] 2> 526731 INFO (zkConnectionManagerCallback-943-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 526734 INFO (zkConnectionManagerCallback-945-thread-1-processing-n:127.0.0.1:54371_solr) [n:127.0.0.1:54371_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 526737 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 526738 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 526739 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54371_solr
[junit4] 2> 526740 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:38040_solr) [n:127.0.0.1:38040_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526740 INFO (zkCallback-921-thread-2-processing-n:127.0.0.1:49142_solr) [n:127.0.0.1:49142_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526740 INFO (zkCallback-917-thread-2-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526740 INFO (zkCallback-919-thread-2-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526751 INFO (zkCallback-935-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526759 INFO (zkCallback-944-thread-1-processing-n:127.0.0.1:54371_solr) [n:127.0.0.1:54371_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526769 INFO (zkCallback-935-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 526813 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54371.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 526822 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54371.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 526822 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54371.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@149bec54
[junit4] 2> 526823 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[1C5A1C03E5256FF7]) [n:127.0.0.1:54371_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-001/node5/.
[junit4] 2> 526873 INFO (zkConnectionManagerCallback-949-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 526875 INFO (qtp691893789-4842) [n:127.0.0.1:35232_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&autoAddReplicas=false&name=MoveReplicaHDFSTest_failed_coll_true&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 526887 INFO (OverseerCollectionConfigSetProcessor-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 526887 INFO (OverseerThreadFactory-1916-thread-2-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_failed_coll_true
[junit4] 2> 526994 INFO (OverseerStateUpdate-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38040/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 526996 INFO (OverseerStateUpdate-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36339/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 526998 INFO (OverseerStateUpdate-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35232/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 526999 INFO (OverseerStateUpdate-73281586938314763-127.0.0.1:35232_solr-n_0000000000) [n:127.0.0.1:35232_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54371/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 527203 INFO (qtp691893789-4845) [n:127.0.0.1:35232_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 527204 INFO (qtp1611642053-4825) [n:127.0.0.1:38040_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 527204 INFO (qtp1611642053-4825) [n:127.0.0.1:38040_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 527204 INFO (qtp691893789-4845) [n:127.0.0.1:35232_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 527204 INFO (qtp820340600-5094) [n:127.0.0.1:54371_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 527204 INFO (qtp820340600-5094) [n:127.0.0.1:54371_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 527212 INFO (qtp719320507-4829) [n:127.0.0.1:36339_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 527213 INFO (qtp719320507-4829) [n:127.0.0.1:36339_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 527317 INFO (zkCallback-919-thread-2-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527317 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:38040_solr) [n:127.0.0.1:38040_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527318 INFO (zkCallback-944-thread-1-processing-n:127.0.0.1:54371_solr) [n:127.0.0.1:54371_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527318 INFO (zkCallback-917-thread-2-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527318 INFO (zkCallback-917-thread-1-processing-n:127.0.0.1:35232_solr) [n:127.0.0.1:35232_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527319 INFO (zkCallback-919-thread-1-processing-n:127.0.0.1:36339_solr) [n:127.0.0.1:36339_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 527319 INFO (zkCallback-944-thread-2-processing-n:127.0.0.1:54371_solr) [n:127.0.0.1:54371_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveRepli
[...truncated too long message...]
en(DFSClient.java:808)
[junit4] 2> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2083)
[junit4] 2> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2069)
[junit4] 2> at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:791)
[junit4] 2> at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
[junit4] 2> at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
[junit4] 2> at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
[junit4] 2> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
[junit4] 2> at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
[junit4] 2> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1517)
[junit4] 2> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1557)
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:527)
[junit4] 2> at org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$21(SolrCore.java:3037)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 603593 ERROR (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 603593 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60863 60863
[junit4] 2> 603696 INFO (Thread-1096) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60863 60863
[junit4] 2> 603697 WARN (Thread-1096) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/collections/MoveReplicaHDFSTest_coll_false/terms/shard2
[junit4] 2> 18 /solr/configs/conf1
[junit4] 2> 14 /solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard1
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 7 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard1
[junit4] 2> 7 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard2
[junit4] 2> 7 /solr/collections/MoveReplicaHDFSTest_coll_false/terms/shard1
[junit4] 2> 7 /solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard2
[junit4] 2> 6 /solr/security.json
[junit4] 2> 5 /solr/collections/MoveReplicaHDFSTest_failed_coll_false/terms/shard2
[junit4] 2> 5 /solr/collections/MoveReplicaHDFSTest_failed_coll_false/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 66 /solr/collections/MoveReplicaHDFSTest_coll_false/state.json
[junit4] 2> 58 /solr/collections/MoveReplicaHDFSTest_coll_true/state.json
[junit4] 2> 39 /solr/collections/MoveReplicaHDFSTest_failed_coll_false/state.json
[junit4] 2> 37 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/state.json
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 3 /solr/collections/MoveReplicaHDFSTest_coll_false/leader_elect/shard2/election/73281586938314761-core_node8-n_0000000000
[junit4] 2> 3 /solr/overseer_elect/election/73281586938314762-127.0.0.1:36339_solr-n_0000000001
[junit4] 2> 3 /solr/collections/MoveReplicaHDFSTest_coll_true/leader_elect/shard1/election/73281586938314761-core_node5-n_0000000000
[junit4] 2> 2 /solr/autoscaling.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 31 /solr/collections
[junit4] 2> 19 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 603698 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 603734 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 603740 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 603740 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1709940034-10.41.0.5-1518424741939 (Datanode Uuid 514e0124-c7a8-47ae-a4b3-2b5144910f16) service to localhost/127.0.0.1:50267
[junit4] 2> 603746 WARN (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 603784 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 603889 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 603889 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50267) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1709940034-10.41.0.5-1518424741939 (Datanode Uuid 102aab93-1c54-47bb-877e-04628b4564bd) service to localhost/127.0.0.1:50267
[junit4] 2> 603904 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 604025 INFO (SUITE-MoveReplicaHDFSTest-seed#[1C5A1C03E5256FF7]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60863 60863
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_1C5A1C03E5256FF7-001
[junit4] 2> Feb 12, 2018 8:40:21 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=Lucene50(blocksize=128), id=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=563, maxMBSortInHeap=5.497972874924903, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-SA, timezone=Pacific/Samoa
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=254487160,total=517996544
[junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, TestHashQParserPlugin, TestFieldSortValues, SpellPossibilityIteratorTest, PreAnalyzedUpdateProcessorTest, DocExpirationUpdateProcessorFactoryTest, CopyFieldTest, TestRestManager, AnalyticsQueryTest, TestSolrConfigHandler, TestClusterStateProvider, TestPerFieldSimilarityWithDefaultOverride, TestFaceting, BlobRepositoryCloudTest, SolrGangliaReporterTest, TestJmxIntegration, HttpSolrCallGetCoreTest, NumericFieldsTest, TestLegacyFieldCache, HighlighterMaxOffsetTest, HdfsDirectoryTest, TestTrieFacet, MoreLikeThisHandlerTest, PKIAuthenticationIntegrationTest, TestClusterProperties, MetricUtilsTest, CollectionsAPIAsyncDistributedZkTest, TestRealTimeGet, BitVectorTest, PreAnalyzedFieldTest, HdfsRecoverLeaseTest, TestLMJelinekMercerSimilarityFactory, TestClassicSimilarityFactory, SearchHandlerTest, TestCloudPseudoReturnFields, SuggesterTSTTest, SpellCheckComponentTest, SystemLogListenerTest, TestSQLHandlerNonCloud, TestSchemaNameResource, CollectionsAPISolrJTest, ClusterStateTest, TestLockTree, TestBadConfig, UUIDUpdateProcessorFallbackTest, LargeFieldTest, BasicDistributedZk2Test, RecoveryAfterSoftCommitTest, TestAtomicUpdateErrorCases, TestUtilizeNode, TestConfigsApi, OverseerCollectionConfigSetProcessorTest, WordBreakSolrSpellCheckerTest, TestCustomStream, TestNamedUpdateProcessors, DistributedFacetSimpleRefinementLongTailTest, MoveReplicaHDFSTest]
[junit4] Completed [174/775 (2!)] on J2 in 89.41s, 4 tests, 1 error <<< FAILURES!
[...truncated 50502 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 374 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/374/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([186CEA76261C7990:A260DDF979F4AFDF]:0)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1565)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14143 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 2568929 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/init-core-data-001
[junit4] 2> 2568930 WARN (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
[junit4] 2> 2568930 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2568931 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2568931 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001
[junit4] 2> 2568932 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2568932 INFO (Thread-8280) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2568932 INFO (Thread-8280) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2568934 ERROR (Thread-8280) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2569032 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:58810
[junit4] 2> 2569035 INFO (zkConnectionManagerCallback-9303-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569037 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1045900e23e0000, likely client has closed socket
[junit4] 2> 2569043 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: २०१७-११-२२T०२:२७:३७+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2569044 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: २०१७-११-२२T०२:२७:३७+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2569044 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2569044 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2569044 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ade2e83{/solr,null,AVAILABLE}
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@741c535b{HTTP/1.1,[http/1.1]}{127.0.0.1:35296}
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-2) [ ] o.e.j.s.Server Started @2571158ms
[junit4] 2> 2569045 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35296}
[junit4] 2> 2569046 ERROR (jetty-launcher-9300-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a8f3286{/solr,null,AVAILABLE}
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T07:39:48.776Z
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7c2b184e{HTTP/1.1,[http/1.1]}{127.0.0.1:42776}
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-1) [ ] o.e.j.s.Server Started @2571159ms
[junit4] 2> 2569046 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42776}
[junit4] 2> 2569046 ERROR (jetty-launcher-9300-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2569047 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2569047 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2569047 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2569047 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T07:39:48.777Z
[junit4] 2> 2569047 INFO (zkConnectionManagerCallback-9305-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569048 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2569053 INFO (zkConnectionManagerCallback-9307-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569053 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2569056 INFO (jetty-launcher-9300-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58810/solr
[junit4] 2> 2569057 INFO (zkConnectionManagerCallback-9311-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569059 INFO (zkConnectionManagerCallback-9313-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569060 INFO (jetty-launcher-9300-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58810/solr
[junit4] 2> 2569061 INFO (zkConnectionManagerCallback-9317-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569064 INFO (zkConnectionManagerCallback-9319-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569129 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2569129 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2569130 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35296_solr
[junit4] 2> 2569130 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42776_solr
[junit4] 2> 2569131 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.Overseer Overseer (id=73281354275356676-127.0.0.1:35296_solr-n_0000000000) starting
[junit4] 2> 2569139 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2569140 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2569144 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35296_solr
[junit4] 2> 2569151 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2569151 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2569151 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2569152 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2569152 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2569152 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2569153 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2569154 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2569154 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2569155 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2569201 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35296.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569208 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42776.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569208 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35296.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569209 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35296.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569210 INFO (jetty-launcher-9300-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001/node2/.
[junit4] 2> 2569216 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42776.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569216 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42776.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569217 INFO (jetty-launcher-9300-thread-1) [n:127.0.0.1:42776_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001/node1/.
[junit4] 2> 2569258 INFO (zkConnectionManagerCallback-9325-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569260 INFO (zkConnectionManagerCallback-9329-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569261 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2569261 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58810/solr ready
[junit4] 2> 2569280 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 2569281 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 2569281 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 2569281 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 2569283 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2569283 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2569283 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 2569287 INFO (qtp109629244-34002) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569288 INFO (qtp170881004-34006) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569288 DEBUG (qtp109629244-33998) [n:127.0.0.1:35296_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2569289 INFO (qtp109629244-33998) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2569289 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 2569290 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2569290 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2569290 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 2569291 INFO (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2569292 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2569292 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2569292 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2569293 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 2569293 INFO (qtp109629244-34004) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569294 INFO (qtp170881004-34009) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569294 DEBUG (qtp109629244-34000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2569294 INFO (qtp109629244-34000) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2569294 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 2569295 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2569295 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2569295 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 2569295 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2569295 INFO (qtp109629244-34005) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569295 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 2569296 INFO (qtp170881004-34006) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2569296 DEBUG (qtp109629244-34002) [n:127.0.0.1:35296_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2569296 INFO (qtp109629244-34002) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2569296 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 2569297 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: २०१७-११-२२T०२:२७:३७+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2569297 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2569297 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2569297 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 2569298 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2569298 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 2569298 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2569298 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2569298 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2569298 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b1e7967{/solr,null,AVAILABLE}
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@193f74f8{HTTP/1.1,[http/1.1]}{127.0.0.1:37620}
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server Started @2571411ms
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37620}
[junit4] 2> 2569299 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2569299 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T07:39:49.029Z
[junit4] 2> 2569300 INFO (zkConnectionManagerCallback-9331-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569301 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2569307 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58810/solr
[junit4] 2> 2569308 INFO (zkConnectionManagerCallback-9335-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569309 INFO (zkConnectionManagerCallback-9337-thread-1-processing-n:127.0.0.1:37620_solr) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569314 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2569316 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2569317 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37620_solr
[junit4] 2> 2569318 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2569318 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2569323 INFO (zkCallback-9336-thread-1-processing-n:127.0.0.1:37620_solr) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2569327 INFO (zkCallback-9328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2569403 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37620.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569413 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37620.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569414 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37620.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2569415 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:37620_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001/node3/.
[junit4] 2> 2570295 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2570295 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2570295 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:37620_solr at time 18673100688310153
[junit4] 2> 2571295 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2571296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2572296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2572296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2573296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2573296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2573296 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:37620_solr] added at times [18673100688310153], now=18673103689865746
[junit4] 2> 2573297 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4257160d61bf89Ta3yyn8sn5esa1c8o3gnhvauu2",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18673100688310153,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673100688310153],
[junit4] 2> "nodeNames":["127.0.0.1:37620_solr"]}}
[junit4] 2> 2573297 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 2573298 INFO (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2573299 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4257160d61bf89Ta3yyn8sn5esa1c8o3gnhvauu2",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18673100688310153,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673100688310153],
[junit4] 2> "_enqueue_time_":18673103691028707,
[junit4] 2> "nodeNames":["127.0.0.1:37620_solr"]}}
[junit4] 2> 2573301 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 2573301 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=4257160d61bf89Ta3yyn8sn5esa1c8o3gnhvauu2
[junit4] 2> 2575302 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: २०१७-११-२२T०२:२७:३७+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2575304 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2575304 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2575304 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2575305 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ba382d9{/solr,null,AVAILABLE}
[junit4] 2> 2575305 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5531e9f7{HTTP/1.1,[http/1.1]}{127.0.0.1:33941}
[junit4] 2> 2575305 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server Started @2577418ms
[junit4] 2> 2575305 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33941}
[junit4] 2> 2575306 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2575306 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2575306 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2575306 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2575307 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T07:39:55.037Z
[junit4] 2> 2575322 INFO (zkConnectionManagerCallback-9341-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2575323 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2575334 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58810/solr
[junit4] 2> 2575335 INFO (zkConnectionManagerCallback-9345-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2575337 INFO (zkConnectionManagerCallback-9347-thread-1-processing-n:127.0.0.1:33941_solr) [n:127.0.0.1:33941_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2575345 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2575347 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2575350 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33941_solr
[junit4] 2> 2575352 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2575352 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2575352 INFO (zkCallback-9328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2575359 INFO (zkCallback-9336-thread-1-processing-n:127.0.0.1:37620_solr) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2575361 INFO (zkCallback-9346-thread-1-processing-n:127.0.0.1:33941_solr) [n:127.0.0.1:33941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2575426 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33941.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2575436 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33941.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2575436 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33941.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2575437 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [n:127.0.0.1:33941_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001/node4/.
[junit4] 2> 2578301 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2578301 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2578302 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:33941_solr at time 18673108695005861
[junit4] 2> 2579303 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2579303 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2580303 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2580304 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2581304 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2581304 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2581304 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:33941_solr] added at times [18673108695005861], now=18673111697730443
[junit4] 2> 2581304 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"425717ea9e3aa5Ta3yyn8sn5esa1c8o3gnhvauu4",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18673108695005861,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673108695005861],
[junit4] 2> "nodeNames":["127.0.0.1:33941_solr"]}}
[junit4] 2> 2581305 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 2581306 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"425717ea9e3aa5Ta3yyn8sn5esa1c8o3gnhvauu4",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18673108695005861,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673108695005861],
[junit4] 2> "_enqueue_time_":18673111698190053,
[junit4] 2> "nodeNames":["127.0.0.1:33941_solr"]}}
[junit4] 2> 2581306 WARN (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: test1 for trigger event: {
[junit4] 2> "id":"425717ea9e3aa5Ta3yyn8sn5esa1c8o3gnhvauu4",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18673108695005861,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673108695005861],
[junit4] 2> "_enqueue_time_":18673111698190053,
[junit4] 2> "nodeNames":["127.0.0.1:33941_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:303)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction.process(TriggerIntegrationTest.java:989)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:300)
[junit4] 2> ... 6 more
[junit4] 2> 2581307 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 2581307 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=425717ea9e3aa5Ta3yyn8sn5esa1c8o3gnhvauu4
[junit4] 2> 2583306 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[186CEA76261C7990]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 2583344 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 2583345 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 2583345 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 2583345 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 2583351 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2583351 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2583351 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 2583355 INFO (qtp555842462-34092) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583357 INFO (qtp109629244-34002) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583358 INFO (qtp58953156-34062) [n:127.0.0.1:37620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583363 INFO (qtp170881004-34006) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583363 DEBUG (qtp555842462-34088) [n:127.0.0.1:33941_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2583364 INFO (qtp555842462-34088) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 2583364 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 2583366 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2583366 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2583366 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 2583366 INFO (qtp555842462-34093) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583370 INFO (qtp109629244-34000) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583371 INFO (qtp58953156-34064) [n:127.0.0.1:37620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583371 INFO (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2583372 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2583372 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 2583372 INFO (qtp170881004-34009) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2583372 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2583373 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2583373 DEBUG (qtp555842462-34090) [n:127.0.0.1:33941_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2583373 INFO (qtp555842462-34090) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 2583374 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 2583374 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2583374 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2583375 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2583375 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2583375 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 2583377 INFO (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2583378 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2583378 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: २०१७-११-२२T०२:२७:३७+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2583378 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2583378 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2583378 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 2583379 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2583379 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2583379 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6e01b2a7{/solr,null,AVAILABLE}
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@56f119aa{HTTP/1.1,[http/1.1]}{127.0.0.1:33021}
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.Server Started @2585493ms
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33021}
[junit4] 2> 2583380 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2583380 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2583381 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2583381 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T07:40:03.111Z
[junit4] 2> 2583382 INFO (zkConnectionManagerCallback-9351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2583382 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2583389 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58810/solr
[junit4] 2> 2583390 INFO (zkConnectionManagerCallback-9355-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2583392 INFO (zkConnectionManagerCallback-9357-thread-1-processing-n:127.0.0.1:33021_solr) [n:127.0.0.1:33021_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2583404 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2583407 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2583409 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33021_solr
[junit4] 2> 2583411 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583411 INFO (zkCallback-9328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583411 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583411 INFO (zkCallback-9336-thread-1-processing-n:127.0.0.1:37620_solr) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583411 INFO (zkCallback-9346-thread-1-processing-n:127.0.0.1:33941_solr) [n:127.0.0.1:33941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583417 INFO (zkCallback-9356-thread-1-processing-n:127.0.0.1:33021_solr) [n:127.0.0.1:33021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2583511 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2583520 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2583520 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dedff02
[junit4] 2> 2583522 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [n:127.0.0.1:33021_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001/tempDir-001/node5/.
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:33021_solr at time 18673114767429012
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:33021_solr] added at times [18673114767429012], now=18673114767502241
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42571954900d94Ta3yyn8sn5esa1c8o3gnhvauu5",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18673114767429012,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673114767429012],
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2584374 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 2584375 INFO (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2584376 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42571954900d94Ta3yyn8sn5esa1c8o3gnhvauu5",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18673114767429012,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673114767429012],
[junit4] 2> "_enqueue_time_":18673114767760950,
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2584376 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger1
[junit4] 2> 2584377 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2584377 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2584377 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=42571954900d94Ta3yyn8sn5esa1c8o3gnhvauu5
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:33021_solr at time 18673119773536367
[junit4] 2> 2589380 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:33021_solr] added at times [18673119773536367], now=18673119773754674
[junit4] 2> 2589381 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42571a7ef3306fTa3yyn8sn5esa1c8o3gnhvauu7",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18673119773536367,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673119773536367],
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2589381 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 2589382 INFO (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2589382 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42571a7ef3306fTa3yyn8sn5esa1c8o3gnhvauu7",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18673119773536367,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673119773536367],
[junit4] 2> "_enqueue_time_":18673119774339591,
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2589382 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518421204106000000 current time = 1518421209112000000
[junit4] 2> real diff: 5006000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 2589383 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger2
[junit4] 2> 2589389 INFO (qtp1142932834-34118) [n:127.0.0.1:33021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589390 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2589390 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2589390 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 8 ms for event id=42571a7ef3306fTa3yyn8sn5esa1c8o3gnhvauu7
[junit4] 2> 2589390 INFO (qtp555842462-34090) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589393 INFO (qtp109629244-34000) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589394 INFO (qtp58953156-34064) [n:127.0.0.1:37620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589397 INFO (qtp170881004-34009) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589397 DEBUG (qtp1142932834-34114) [n:127.0.0.1:33021_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2589398 INFO (qtp1142932834-34114) [n:127.0.0.1:33021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=12
[junit4] 2> 2589399 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 2589399 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589399 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2589401 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589401 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589401 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2589401 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 2589403 INFO (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2589404 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 2589404 DEBUG (ScheduledTrigger-8434-thread-3) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 5
[junit4] 2> 2589411 INFO (qtp1142932834-34118) [n:127.0.0.1:33021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589413 INFO (qtp555842462-34094) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589415 INFO (qtp109629244-34002) [n:127.0.0.1:35296_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589417 INFO (qtp58953156-34066) [n:127.0.0.1:37620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589419 INFO (qtp170881004-34006) [n:127.0.0.1:42776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2589419 DEBUG (qtp555842462-34092) [n:127.0.0.1:33941_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2589420 INFO (qtp555842462-34092) [n:127.0.0.1:33941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 2589420 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 2589421 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589421 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2589421 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589421 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589421 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2589422 DEBUG (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37620_solr, 127.0.0.1:33021_solr, 127.0.0.1:33941_solr, 127.0.0.1:35296_solr, 127.0.0.1:42776_solr]
[junit4] 2> 2589422 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 2589424 INFO (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2589424 DEBUG (OverseerAutoScalingTriggerThread-73281354275356676-127.0.0.1:35296_solr-n_0000000000) [n:127.0.0.1:35296_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 2589425 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 5
[junit4] 2> 2589425 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@56f119aa{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2589425 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=655875618
[junit4] 2> 2589426 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2589426 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@110c2264: rootName = solr_33021, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@d8ad16b
[junit4] 2> 2589435 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2589435 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@37af00a9: rootName = solr_33021, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@46efc70e
[junit4] 2> 2589440 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2589440 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@651f747e: rootName = solr_33021, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@21c6bcb0
[junit4] 2> 2589443 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33021_solr
[junit4] 2> 2589444 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2589444 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2589444 INFO (zkCallback-9346-thread-1-processing-n:127.0.0.1:33941_solr) [n:127.0.0.1:33941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589444 INFO (zkCallback-9356-thread-1-processing-n:127.0.0.1:33021_solr) [n:127.0.0.1:33021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589444 INFO (zkCallback-9328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589445 INFO (zkCallback-9318-thread-1-processing-n:127.0.0.1:42776_solr) [n:127.0.0.1:42776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589445 INFO (zkCallback-9336-thread-1-processing-n:127.0.0.1:37620_solr) [n:127.0.0.1:37620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589445 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6e01b2a7{/solr,null,UNAVAILABLE}
[junit4] 2> 2589445 INFO (zkCallback-9312-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2589445 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2590408 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 4
[junit4] 2> 2590408 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:33021_solr
[junit4] 2> 2590408 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:33021_solr]
[junit4] 2> 2590408 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42571abc3d42d6Ta3yyn8sn5esa1c8o3gnhvauu9",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18673120801800918,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673120801800918],
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2590409 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 2590409 INFO (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2590410 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42571abc3d42d6Ta3yyn8sn5esa1c8o3gnhvauu9",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18673120801800918,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673120801800918],
[junit4] 2> "_enqueue_time_":18673120802136765,
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2590410 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger1
[junit4] 2> 2590411 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 2590411 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2590411 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 2590411 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2590411 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=42571abc3d42d6Ta3yyn8sn5esa1c8o3gnhvauu9
[junit4] 2> 2595411 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2595412 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 4
[junit4] 2> 2595412 DEBUG (ScheduledTrigger-8434-thread-2) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-4) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 4
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:33021_solr
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:33021_solr]
[junit4] 2> 2595413 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42571be68e0a31Ta3yyn8sn5esa1c8o3gnhvauub",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18673125806705201,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673125806705201],
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2595414 DEBUG (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 2595415 INFO (ScheduledTrigger-8434-thread-1) [n:127.0.0.1:35296_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2595415 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42571be68e0a31Ta3yyn8sn5esa1c8o3gnhvauub",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18673125806705201,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18673125806705201],
[junit4] 2> "_enqueue_time_":18673125807184478,
[junit4] 2> "nodeNames":["127.0.0.1:33021_solr"]}}
[junit4] 2> 2595415 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518421210140000000 current time = 1518421215145000000
[junit4] 2> real diff: 5005000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 2595415 INFO (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger2
[junit4] 2> 2595415 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[186CEA76261C7990]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 2595419 DEBUG (AutoscalingActionExecutor-8435-thread-1-processing-n:127.0.0.1:35296_solr) [n:127.0.0.1:35296_solr ] o.a.s.c.a.ScheduledTriggers Resuming tr
[...truncated too long message...]
expired for /autoscaling/nodeLost/127.0.0.1:57060_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:882)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2770059 INFO (jetty-closer-9301-thread-12) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@45e7e516{/solr,null,UNAVAILABLE}
[junit4] 2> 2770059 INFO (jetty-closer-9301-thread-12) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2770060 ERROR (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2770060 INFO (SUITE-TriggerIntegrationTest-seed#[186CEA76261C7990]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58810 58810
[junit4] 2> 2770155 INFO (Thread-8280) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58810 58810
[junit4] 2> 2770155 WARN (Thread-8280) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 41 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 8 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 132 /solr/collections/collection1/state.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 13 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 4 /solr/overseer_elect/election/73281354275356716-127.0.0.1:49362_solr-n_0000000013
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/73281354275356710-core_node3-n_0000000000
[junit4] 2> 3 /solr/overseer_elect/election/73281354275356686-127.0.0.1:33941_solr-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73281354275356710-127.0.0.1:41760_solr-n_0000000011
[junit4] 2> 2 /solr/overseer_elect/election/73281354275356698-127.0.0.1:39986_solr-n_0000000007
[junit4] 2> 2 /solr/overseer_elect/election/73281354275356707-127.0.0.1:60573_solr-n_0000000010
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73281354275356716-core_node8-n_0000000003
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_186CEA76261C7990-001
[junit4] 2> Feb 12, 2018 7:43:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Direct), x_s=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1489, maxMBSortInHeap=6.244574450600561, sim=RandomSimilarity(queryNorm=false): {}, locale=mt, timezone=Asia/Colombo
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=174054768,total=476577792
[junit4] 2> NOTE: All tests run in this JVM: [BlockJoinFacetRandomTest, FileUtilsTest, TestSolrConfigHandlerConcurrent, TestExceedMaxTermLength, HighlighterTest, TestDistribStateManager, SolrCloudExampleTest, TestTolerantUpdateProcessorRandomCloud, TestJettySolrRunner, SegmentsInfoRequestHandlerTest, SubstringBytesRefFilterTest, SolrCoreTest, TestQueryUtils, TestImplicitCoreProperties, SchemaVersionSpecificBehaviorTest, SliceStateTest, ParsingFieldUpdateProcessorsTest, TestSlowCompositeReaderWrapper, TestReplicaProperties, DirectoryFactoryTest, TestSimpleTextCodec, CursorMarkTest, TestSQLHandler, ResourceLoaderTest, PathHierarchyTokenizerFactoryTest, PrimUtilsTest, SearchRateTriggerTest, ZkControllerTest, TestDocTermOrds, StressHdfsTest, TestPullReplicaErrorHandling, TolerantUpdateProcessorTest, SortByFunctionTest, TestRecoveryHdfs, TestFieldCacheWithThreads, PrimitiveFieldTypeTest, TestCloudPivotFacet, TestMultiWordSynonyms, TestFieldTypeResource, TestXmlQParserPlugin, TestTrie, DistributedMLTComponentTest, QueryEqualityTest, TestSolrIndexConfig, TestEmbeddedSolrServerAdminHandler, TestReversedWildcardFilterFactory, TestRTimerTree, DocValuesMissingTest, TestManagedSchema, TestExactStatsCache, TestStreamBody, CdcrVersionReplicationTest, CollectionStateFormat2Test, TestZkChroot, ChaosMonkeySafeLeaderTest, HdfsBasicDistributedZkTest, JsonLoaderTest, TestManagedResourceStorage, TestReplicationHandler, TestDistributedGrouping, TestSortByMinMaxFunction, TestBlendedInfixSuggestions, BufferStoreTest, TestSkipOverseerOperations, QueryResultKeyTest, ClassificationUpdateProcessorFactoryTest, TestCursorMarkWithoutUniqueKey, DistribJoinFromCollectionTest, TestSimpleTrackingShardHandler, TestSolrCloudWithSecureImpersonation, TestCustomSort, HdfsNNFailoverTest, TestStressUserVersions, DistributedQueryComponentCustomSortTest, TestHttpShardHandlerFactory, SparseHLLTest, BasicFunctionalityTest, PeerSyncTest, ChangedSchemaMergeTest, HdfsRestartWhileUpdatingTest, SignatureUpdateProcessorFactoryTest, TestConfigReload, OverriddenZkACLAndCredentialsProvidersTest, FastVectorHighlighterTest, BaseCdcrDistributedZkTest, CollectionTooManyReplicasTest, TestLegacyNumericRangeQueryBuilder, CachingDirectoryFactoryTest, TemplateUpdateProcessorTest, DocumentBuilderTest, TestNumericRangeQuery32, TestCollapseQParserPlugin, PreAnalyzedFieldManagedSchemaCloudTest, JavabinLoaderTest, TestSurroundQueryParser, RulesTest, DeleteNodeTest, TestCollectionsAPIViaSolrCloudCluster, AlternateDirectoryTest, TermsComponentTest, TestSolrCloudSnapshots, TestCustomDocTransformer, DirectSolrConnectionTest, QueryParsingTest, TestNumericTerms64, TestRandomCollapseQParserPlugin, PeerSyncWithIndexFingerprintCachingTest, SolrJmxReporterCloudTest, TestSystemIdResolver, ResponseBuilderTest, TestOnReconnectListenerSupport, DistributedIntervalFacetingTest, ConjunctionSolrSpellCheckerTest, TestCloudDeleteByQuery, ShardRoutingCustomTest, ConcurrentDeleteAndCreateCollectionTest, CloneFieldUpdateProcessorFactoryTest, V2StandaloneTest, SuggestComponentTest, SolrCloudReportersTest, TestSolrJ, SimplePostToolTest, AsyncCallRequestStatusResponseTest, SystemInfoHandlerTest, TestFoldingMultitermQuery, TestGraphTermsQParserPlugin, FieldMutatingUpdateProcessorTest, HdfsTlogReplayBufferedWhileIndexingTest, TestRebalanceLeaders, BasicDistributedZkTest, LegacyCloudClusterPropTest, MoveReplicaHDFSFailoverTest, RestartWhileUpdatingTest, SaslZkACLProviderTest, TestCloudSearcherWarming, TestDistribDocBasedVersion, TestLockTree, TestPrepRecovery, TestSizeLimitedDistributedMap, TestStressLiveNodes, TestTlogReplica, CustomCollectionTest, HdfsCollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, TestCollectionAPI, AutoAddReplicasPlanActionTest, AutoScalingHandlerTest, ComputePlanActionTest, MetricTriggerTest, NodeAddedTriggerTest, NodeLostTriggerTest, TestPolicyCloud, TriggerIntegrationTest]
[junit4] Completed [607/775 (1!)] on J1 in 201.78s, 14 tests, 1 error <<< FAILURES!
[...truncated 49023 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 373 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/373/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.facet.PivotFacetTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.PivotFacetTest: 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.PivotFacetTest:
1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([6F7551C79559B0BE]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.facet.PivotFacetTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([6F7551C79559B0BE]:0)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 8 in http://127.0.0.1:36482/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 8 in http://127.0.0.1:36482/solr
at __randomizedtesting.SeedInfo.seed([8A984FB4814D4122:4B683618AC1D8B85]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:885)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([8A984FB4814D4122:BB262250FFE734D0]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown(TriggerIntegrationTest.java:1177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14065 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTlogReplica
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/init-core-data-001
[junit4] 2> 4267894 WARN (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
[junit4] 2> 4267895 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 4267896 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 4267896 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 4267897 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001
[junit4] 2> 4267897 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4267897 INFO (Thread-3743) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4267897 INFO (Thread-3743) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4267899 ERROR (Thread-3743) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4267997 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37101
[junit4] 2> 4267999 INFO (zkConnectionManagerCallback-4962-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268002 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4268003 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4268003 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4268003 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4268003 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7638c8ab{/solr,null,AVAILABLE}
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7ab89d82{HTTP/1.1,[http/1.1]}{127.0.0.1:39760}
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-1) [ ] o.e.j.s.Server Started @4279021ms
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39760}
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b4f796b{/solr,null,AVAILABLE}
[junit4] 2> 4268004 ERROR (jetty-launcher-4959-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4268004 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@544d32b2{HTTP/1.1,[http/1.1]}{127.0.0.1:36482}
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.e.j.s.Server Started @4279022ms
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36482}
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T01:57:16.537Z
[junit4] 2> 4268005 ERROR (jetty-launcher-4959-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4268005 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T01:57:16.537Z
[junit4] 2> 4268006 INFO (zkConnectionManagerCallback-4964-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268006 INFO (zkConnectionManagerCallback-4966-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268007 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 4268007 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 4268015 INFO (jetty-launcher-4959-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37101/solr
[junit4] 2> 4268015 INFO (jetty-launcher-4959-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37101/solr
[junit4] 2> 4268027 INFO (zkConnectionManagerCallback-4971-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268028 INFO (zkConnectionManagerCallback-4974-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268030 INFO (zkConnectionManagerCallback-4976-thread-1-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268030 INFO (zkConnectionManagerCallback-4978-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268121 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4268121 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4268122 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39760_solr
[junit4] 2> 4268123 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36482_solr
[junit4] 2> 4268123 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.c.Overseer Overseer (id=72105962054811653-127.0.0.1:39760_solr-n_0000000000) starting
[junit4] 2> 4268124 INFO (zkCallback-4975-thread-1-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4268129 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4268135 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39760_solr
[junit4] 2> 4268137 INFO (zkCallback-4975-thread-1-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4268144 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4268191 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36482.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268195 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39760.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268200 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36482.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268200 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36482.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268201 INFO (jetty-launcher-4959-thread-2) [n:127.0.0.1:36482_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node2/.
[junit4] 2> 4268204 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39760.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268204 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39760.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4268205 INFO (jetty-launcher-4959-thread-1) [n:127.0.0.1:39760_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node1/.
[junit4] 2> 4268251 INFO (zkConnectionManagerCallback-4984-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268254 INFO (zkConnectionManagerCallback-4988-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4268255 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 4268256 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37101/solr ready
[junit4] 2> 4268259 INFO (SUITE-TestTlogReplica-seed#[8A984FB4814D4122]-worker) [ ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
[junit4] 2> 4268260 INFO (qtp1809463584-23175) [n:127.0.0.1:36482_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4268267 INFO (qtp1809463584-23175) [n:127.0.0.1:36482_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 4268285 INFO (TEST-TestTlogReplica.testOnlyLeaderIndexes-seed#[8A984FB4814D4122]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOnlyLeaderIndexes
[junit4] 2> 4268286 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_only_leader_indexes&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4268288 INFO (OverseerThreadFactory-6795-thread-1-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_only_leader_indexes
[junit4] 2> 4268395 INFO (OverseerStateUpdate-72105962054811653-127.0.0.1:39760_solr-n_0000000000) [n:127.0.0.1:39760_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_only_leader_indexes",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_only_leader_indexes_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36482/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 4268397 INFO (OverseerStateUpdate-72105962054811653-127.0.0.1:39760_solr-n_0000000000) [n:127.0.0.1:39760_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_only_leader_indexes",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_only_leader_indexes_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39760/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 4268599 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 4268600 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 4268600 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 4268600 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 4268703 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4268703 INFO (zkCallback-4975-thread-1-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4268703 INFO (zkCallback-4977-thread-2-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4268704 INFO (zkCallback-4975-thread-2-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4269612 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 4269612 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 4269624 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Schema name=minimal
[junit4] 2> 4269628 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4269628 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_only_leader_indexes_shard1_replica_t2' using configuration from collection tlog_replica_test_only_leader_indexes, trusted=true
[junit4] 2> 4269629 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39760.solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4269629 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4269629 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_only_leader_indexes_shard1_replica_t2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node1/tlog_replica_test_only_leader_indexes_shard1_replica_t2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node1/./tlog_replica_test_only_leader_indexes_shard1_replica_t2/data/]
[junit4] 2> 4269632 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Schema name=minimal
[junit4] 2> 4269634 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4269634 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_only_leader_indexes_shard1_replica_t1' using configuration from collection tlog_replica_test_only_leader_indexes, trusted=true
[junit4] 2> 4269635 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36482.solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3291d2e7
[junit4] 2> 4269635 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4269635 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_only_leader_indexes_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node2/tlog_replica_test_only_leader_indexes_shard1_replica_t1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_8A984FB4814D4122-001/tempDir-001/node2/./tlog_replica_test_only_leader_indexes_shard1_replica_t1/data/]
[junit4] 2> 4269710 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4269710 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4269710 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4269710 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4269711 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4269711 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4269711 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4269711 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4269713 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7dc889bc[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
[junit4] 2> 4269713 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7aa5a5ec[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main]
[junit4] 2> 4269714 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 4269714 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 4269715 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4269715 INFO (searcherExecutor-6801-thread-1-processing-n:127.0.0.1:36482_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t1 s:shard1 c:tlog_replica_test_only_leader_indexes r:core_node3) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Registered new searcher Searcher@7aa5a5ec[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4269716 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592158467651534848
[junit4] 2> 4269719 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 4269720 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 4269720 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4269720 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592158467655729152
[junit4] 2> 4269723 INFO (searcherExecutor-6800-thread-1-processing-n:127.0.0.1:39760_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t2 s:shard1 c:tlog_replica_test_only_leader_indexes r:core_node4) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@7dc889bc[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4269727 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4269727 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4269727 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/
[junit4] 2> 4269727 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_only_leader_indexes_shard1_replica_t1 url=http://127.0.0.1:36482/solr START replicas=[http://127.0.0.1:39760/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t2/] nUpdates=100
[junit4] 2> 4269729 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4269828 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4269828 INFO (zkCallback-4977-thread-2-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4269828 INFO (zkCallback-4975-thread-3-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4269828 INFO (zkCallback-4975-thread-2-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270028 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_only_leader_indexes_shard1_replica_t1 url=http://127.0.0.1:36482/solr DONE. We have no versions. sync failed.
[junit4] 2> 4270028 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 4270028 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 4270028 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4270028 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_only_leader_indexes_shard1_replica_t1 stopping background replication from leader
[junit4] 2> 4270031 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/ shard1
[junit4] 2> 4270133 INFO (zkCallback-4977-thread-2-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270133 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270133 INFO (zkCallback-4975-thread-3-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270133 INFO (zkCallback-4975-thread-2-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270182 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4270184 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1585
[junit4] 2> 4270284 INFO (zkCallback-4977-thread-2-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270285 INFO (zkCallback-4975-thread-3-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270284 INFO (zkCallback-4975-thread-2-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270284 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270728 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_only_leader_indexes_shard1_replica_t2 starting background replication from leader
[junit4] 2> 4270728 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 4270730 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 4270730 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4270731 INFO (qtp241573580-23167) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2131
[junit4] 2> 4270734 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 4270833 INFO (zkCallback-4975-thread-2-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270833 INFO (zkCallback-4977-thread-2-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270833 INFO (zkCallback-4975-thread-3-processing-n:127.0.0.1:39760_solr) [n:127.0.0.1:39760_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4270833 INFO (zkCallback-4977-thread-1-processing-n:127.0.0.1:36482_solr) [n:127.0.0.1:36482_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4271078 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/
[junit4] 2> 4271079 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 4271080 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 4271080 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 4271080 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 4271080 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 4271080 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 4271082 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@483de4fd[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
[junit4] 2> 4271084 INFO (searcherExecutor-6800-thread-1-processing-n:127.0.0.1:39760_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t2 s:shard1 c:tlog_replica_test_only_leader_indexes r:core_node4) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@483de4fd[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4271734 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_only_leader_indexes&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3448
[junit4] 2> 4271742 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{add=[1 (1592158469771755520), 2 (1592158469774901248), 3 (1592158469774901249), 4 (1592158469774901250)]} 0 0
[junit4] 2> 4271742 INFO (qtp1809463584-23180) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1592158469771755520), 2 (1592158469774901248), 3 (1592158469774901249), 4 (1592158469774901250)]} 0 4
[junit4] 2> 4271745 INFO (qtp1809463584-23180) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271746 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271746 INFO (qtp1809463584-23180) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271747 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271748 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271749 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271749 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271749 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271751 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{delete=[1 (-1592158469784338432)]} 0 0
[junit4] 2> 4271751 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1 (-1592158469784338432)]} 0 1
[junit4] 2> 4271754 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@256e8806[tlog_replica_test_only_leader_indexes_shard1_replica_t1] realtime]
[junit4] 2> 4271756 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@24388850[tlog_replica_test_only_leader_indexes_shard1_replica_t1] realtime]
[junit4] 2> 4271756 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1592158469786435584&distrib.from=http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{deleteByQuery=id:2 (-1592158469786435584)} 0 0
[junit4] 2> 4271756 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=id:2 (-1592158469786435584)} 0 4
[junit4] 2> 4271758 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271759 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271759 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4271760 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 4271761 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1592158469795872768,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 4271761 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@23104b96 commitCommandVersion:1592158469795872768
[junit4] 2> 4271761 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d528c1d[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main]
[junit4] 2> 4271762 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 4271763 INFO (searcherExecutor-6801-thread-1-processing-n:127.0.0.1:36482_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t1 s:shard1 c:tlog_replica_test_only_leader_indexes r:core_node3) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Registered new searcher Searcher@2d528c1d[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C4/2:delGen=2)))}
[junit4] 2> 4271763 INFO (qtp1809463584-23181) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:36482/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
[junit4] 2> 4271763 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_only_leader_indexes:6&commit=true&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 4271765 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
[junit4] 2> 4271767 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4271867 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4271969 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272070 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272171 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272272 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272289 INFO (OverseerCollectionConfigSetProcessor-72105962054811653-127.0.0.1:39760_solr-n_0000000000) [n:127.0.0.1:39760_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 4272373 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272474 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272575 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272676 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272777 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272878 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4272978 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273079 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273180 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273281 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273382 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273483 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273584 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273685 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273786 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273887 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4273988 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4274079 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 4274079 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 4274079 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 1518400640293
[junit4] 2> 4274079 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 4274079 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 4274079 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 4274081 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=1
[junit4] 2> 4274081 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Number of files in latest index in master: 16
[junit4] 2> 4274087 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@4f38af84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@324b5647)
[junit4] 2> 4274087 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_2.liv&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274089 INFO (qtp241573580-23172) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4274094 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274107 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274108 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274109 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274110 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274116 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274117 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274156 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274157 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.mdvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274158 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274159 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.mdvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274159 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274160 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274161 INFO (qtp1809463584-23174) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274162 INFO (qtp1809463584-23177) [n:127.0.0.1:36482_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 4274162 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=2417) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@4f38af84 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@324b5647)
[junit4] 2> 4274171 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 4274172 INFO (indexFetcher-6811-thread-1) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@df3aba4[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
[junit4] 2> 4274173 INFO (searcherExecutor-6800-thread-1-processing-n:127.0.0.1:39760_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t2 s:shard1 c:tlog_replica_test_only_leader_indexes r:core_node4) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@df3aba4[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C4/2:delGen=2)))}
[junit4] 2> 4274190 INFO (qtp241573580-23171) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
[junit4] 2> 4274194 INFO (qtp241573580-23169) [n:127.0.0.1:39760_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLE
[...truncated too long message...]
ackupRestoreCore, TestRestoreCore, CdcrBootstrapTest, TestSolr4Spatial2, TestExactSharedStatsCache, TestRemoteStreaming, TestOrdValues, ZkFailoverTest, TestSolrDeletionPolicy1, TestCloudRecovery, TestDistributedSearch, SuggestComponentContextFilterQueryTest, AtomicUpdatesTest, TestQueryWrapperFilter, TestPHPSerializedResponseWriter, TestCSVLoader, TestSearcherReuse, BooleanFieldTest, SpellCheckCollatorTest, TestBlobHandler, BlockJoinFacetDistribTest, TestWriterPerf, ZkStateReaderTest, MBeansHandlerTest, GraphQueryTest, CursorPagingTest, HdfsWriteToMultipleCollectionsTest, TestExpandComponent, TestSolrCloudWithKerberosAlt, TestConfigSetsAPI, TestGroupingSearch, RequestLoggingTest, ManagedSchemaRoundRobinCloudTest, TestFreeTextSuggestions, DocumentAnalysisRequestHandlerTest, TestConfigOverlay, TestDocBasedVersionConstraints, CloudMLTQParserTest, TestSha256AuthenticationProvider, TestMiniSolrCloudClusterSSL, SolrCLIZkUtilsTest, DistanceUnitsTest, HttpTriggerListenerTest, HdfsChaosMonkeySafeLeaderTest, TestBinaryResponseWriter, SecurityConfHandlerTest, UUIDFieldTest, TestRandomRequestDistribution, BasicDistributedZkTest, TestBadConfig, TestIndexSearcher, HighlighterTest, SimplePostToolTest, TestExtendedDismaxParser, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, TestAtomicUpdateErrorCases, DocValuesMultiTest, SolrInfoBeanTest, IndexBasedSpellCheckerTest, TestValueSourceCache, MoreLikeThisHandlerTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, PingRequestHandlerTest, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestBinaryField, TestSearchPerf, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, EchoParamsTest, TestPerFieldSimilarity, TestFastWriter, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, DateFieldTest, RAMDirectoryFactoryTest, TestLRUCache, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ConnectionReuseTest, AddReplicaTest, AsyncCallRequestStatusResponseTest, ChaosMonkeySafeLeaderWithPullReplicasTest, ConcurrentCreateRoutedAliasTest, CreateCollectionCleanupTest, DeleteNodeTest, DeleteStatusTest, DistribDocExpirationUpdateProcessorTest, HttpPartitionTest, LegacyCloudClusterPropTest, MoveReplicaTest, SSLMigrationTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestPullReplicaErrorHandling, TestRebalanceLeaders, TestShortCircuitedRequests, TestTlogReplica, SimpleCollectionCreateDeleteTest, TestCollectionsAPIViaSolrCloudCluster, TestLocalFSCloudBackupRestore, TestReplicaProperties, AutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest, TriggerIntegrationTest]
[junit4] Completed [649/775 (2!)] on J0 in 194.12s, 14 tests, 1 failure <<< FAILURES!
[...truncated 3529 lines...]
[junit4] Suite: org.apache.solr.analytics.facet.PivotFacetTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analytics/test/J0/temp/solr.analytics.facet.PivotFacetTest_6F7551C79559B0BE-001/init-core-data-001
[junit4] 2> Feb 12, 2018 2:21:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 12, 2018 2:22:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.PivotFacetTest:
[junit4] 2> 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 12, 2018 2:22:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
[junit4] 2> Feb 12, 2018 2:22:12 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, maxDocsPerChunk=909, blockSize=628), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, blockSize=628)), sim=RandomSimilarity(queryNorm=true): {}, locale=es-BO, timezone=Etc/GMT-6
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=176373520,total=331350016
[junit4] 2> NOTE: All tests run in this JVM: [DateMathFunctionTest, LegacyAbstractAnalyticsCloudTest, FilterFunctionTest, LegacyAbstractAnalyticsFacetCloudTest, LegacyNoFacetTest, BottomFunctionTest, CastingStringValueStreamTest, CastingIntValueTest, NegateFunctionTest, PivotFacetTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PivotFacetTest -Dtests.seed=6F7551C79559B0BE -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-BO -Dtests.timezone=Etc/GMT-6 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | PivotFacetTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.analytics.facet.PivotFacetTest:
[junit4] > 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([6F7551C79559B0BE]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=63, name=qtp194080643-63, state=TIMED_WAITING, group=TGRP-PivotFacetTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([6F7551C79559B0BE]:0)
[junit4] Completed [40/78 (1!)] on J0 in 99.78s, 1 test, 2 errors <<< FAILURES!
[...truncated 45398 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 372 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/372/
3 tests failed.
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testMetadata
Error Message:
Error from server at https://127.0.0.1:41161/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:41161/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
at __randomizedtesting.SeedInfo.seed([23295B88B7E20539:2A39118CF00B5950]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:444)
at org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([23295B88B7E20539:27D7F29A8D2887A3]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger(TriggerIntegrationTest.java:521)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([23295B88B7E20539:4ED5FF750DAAFA3E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13057 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> 2383237 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/init-core-data-001
[junit4] 2> 2383239 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
[junit4] 2> 2383239 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2383248 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2383249 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001
[junit4] 2> 2383249 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2383249 INFO (Thread-4363) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2383249 INFO (Thread-4363) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2383358 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:43505
[junit4] 2> 2383358 ERROR (Thread-4363) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2383438 INFO (zkConnectionManagerCallback-2076-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2383586 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2383588 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2383589 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2383602 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2383602 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2383602 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2383603 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c49adfb{/solr,null,AVAILABLE}
[junit4] 2> 2383603 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@17db57b5{SSL,[ssl, http/1.1]}{127.0.0.1:40206}
[junit4] 2> 2383603 INFO (jetty-launcher-2073-thread-1) [ ] o.e.j.s.Server Started @2388475ms
[junit4] 2> 2383603 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40206}
[junit4] 2> 2383604 ERROR (jetty-launcher-2073-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2383604 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2383604 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2383604 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2383604 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:50:10.957Z
[junit4] 2> 2383605 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2383605 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2383605 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2383606 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ccd60b1{/solr,null,AVAILABLE}
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6343d7c8{SSL,[ssl, http/1.1]}{127.0.0.1:33018}
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.e.j.s.Server Started @2388490ms
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33018}
[junit4] 2> 2383619 ERROR (jetty-launcher-2073-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2383619 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:50:10.972Z
[junit4] 2> 2383683 INFO (zkConnectionManagerCallback-2078-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2383684 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2383706 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2383706 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2383706 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2383707 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b931765{/solr,null,AVAILABLE}
[junit4] 2> 2383708 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@73aec599{SSL,[ssl, http/1.1]}{127.0.0.1:37592}
[junit4] 2> 2383720 INFO (jetty-launcher-2073-thread-3) [ ] o.e.j.s.Server Started @2388591ms
[junit4] 2> 2383720 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37592}
[junit4] 2> 2383720 ERROR (jetty-launcher-2073-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2383720 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2383720 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2383720 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2383721 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:50:11.074Z
[junit4] 2> 2383726 INFO (zkConnectionManagerCallback-2082-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2383730 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2383801 INFO (zkConnectionManagerCallback-2080-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2383802 INFO (jetty-launcher-2073-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43505/solr
[junit4] 2> 2383815 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2383956 INFO (jetty-launcher-2073-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43505/solr
[junit4] 2> 2383997 INFO (jetty-launcher-2073-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43505/solr
[junit4] 2> 2384130 INFO (zkConnectionManagerCallback-2086-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384229 INFO (zkConnectionManagerCallback-2090-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384229 INFO (zkConnectionManagerCallback-2094-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384290 INFO (zkConnectionManagerCallback-2096-thread-1-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384334 INFO (zkConnectionManagerCallback-2098-thread-1-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384336 INFO (zkConnectionManagerCallback-2101-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2384787 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2384788 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37592_solr
[junit4] 2> 2384789 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2384789 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer Overseer (id=72104282642382857-127.0.0.1:37592_solr-n_0000000000) starting
[junit4] 2> 2384799 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33018_solr
[junit4] 2> 2384896 INFO (zkCallback-2100-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2384958 INFO (zkCallback-2095-thread-1-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2385004 INFO (zkCallback-2097-thread-1-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2385034 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2385040 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40206_solr
[junit4] 2> 2385058 INFO (zkCallback-2097-thread-1-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2385059 INFO (zkCallback-2100-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2385060 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 2385062 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2385066 INFO (zkCallback-2095-thread-1-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2385095 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:40206_solr"} current state version: 0
[junit4] 2> 2385095 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40206_solr
[junit4] 2> 2385096 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:37592_solr"} current state version: 0
[junit4] 2> 2385096 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:37592_solr
[junit4] 2> 2385103 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37592_solr
[junit4] 2> 2385119 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2385126 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:40206_solr, 127.0.0.1:33018_solr]
[junit4] 2> 2385126 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2385126 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2385126 INFO (zkCallback-2100-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2385126 INFO (zkCallback-2097-thread-1-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2385176 INFO (zkCallback-2095-thread-1-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2385179 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33018.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385188 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2385214 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33018.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385215 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33018.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385215 INFO (jetty-launcher-2073-thread-2) [n:127.0.0.1:33018_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/.
[junit4] 2> 2385221 DEBUG (OverseerAutoScalingTriggerThread-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2385250 DEBUG (ScheduledTrigger-3833-thread-1) [n:127.0.0.1:37592_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2385330 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40206.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385342 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37592.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385374 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40206.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385374 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40206.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385375 INFO (jetty-launcher-2073-thread-1) [n:127.0.0.1:40206_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node1/.
[junit4] 2> 2385378 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37592.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385378 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37592.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b02ed8
[junit4] 2> 2385379 INFO (jetty-launcher-2073-thread-3) [n:127.0.0.1:37592_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/.
[junit4] 2> 2385513 INFO (zkConnectionManagerCallback-2108-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2385522 INFO (zkConnectionManagerCallback-2112-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2385523 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2385524 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43505/solr ready
[junit4] 2> 2385581 INFO (qtp62202932-13301) [n:127.0.0.1:37592_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2385583 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 2385583 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 2385585 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 2385586 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 2385588 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 2385589 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 2385590 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 2385599 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 2385601 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 2385602 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 2385604 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 2385605 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 2385606 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 2385632 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 2385634 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 2385635 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 2385636 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 2385638 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 2385639 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 2385641 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 2385642 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 2385643 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 2385645 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 2385646 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 2385647 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 2385649 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 2385650 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 2385651 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 2385653 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 2385654 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 2385655 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 2385656 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 2385658 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 2385659 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 2385660 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 2385661 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 2385662 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 2385664 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 2385665 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 2385666 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 2385668 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 2385669 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 2385671 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 2385672 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 2385674 INFO (OverseerThreadFactory-3835-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 2385698 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 2385698 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 2385698 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 2385801 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2385801 INFO (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2385802 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 2385802 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2385803 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2385803 INFO (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2385804 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2385804 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2385804 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2385804 INFO (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2385805 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2385805 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2385906 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 2386167 INFO (qtp1526287941-13291) [n:127.0.0.1:33018_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2386168 INFO (qtp1526287941-13291) [n:127.0.0.1:33018_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2386198 INFO (qtp62202932-13305) [n:127.0.0.1:37592_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2386198 INFO (qtp62202932-13305) [n:127.0.0.1:37592_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2386204 INFO (qtp1347073129-13279) [n:127.0.0.1:40206_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2386204 INFO (qtp1347073129-13279) [n:127.0.0.1:40206_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2386213 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "node_name":"127.0.0.1:37592_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2386213 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "node_name":"127.0.0.1:37592_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2386214 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:37592/solr","node_name":"127.0.0.1:37592_solr","state":"down","type":"NRT"}
[junit4] 2> 2386214 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "node_name":"127.0.0.1:37592_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2386218 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "node_name":"127.0.0.1:33018_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2386218 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "node_name":"127.0.0.1:33018_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2386218 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:33018/solr","node_name":"127.0.0.1:33018_solr","state":"down","type":"NRT"}
[junit4] 2> 2386218 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "node_name":"127.0.0.1:33018_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "node_name":"127.0.0.1:37592_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2386222 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "node_name":"127.0.0.1:40206_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2386222 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "node_name":"127.0.0.1:40206_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2386222 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:40206/solr","node_name":"127.0.0.1:40206_solr","state":"down","type":"NRT"}
[junit4] 2> 2386222 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:33018/solr",
[junit4] 2> "node_name":"127.0.0.1:33018_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37592/solr",
[junit4] 2> "node_name":"127.0.0.1:37592_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40206/solr",
[junit4] 2> "node_name":"127.0.0.1:40206_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2386301 DEBUG (ScheduledTrigger-3833-thread-1) [n:127.0.0.1:37592_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2386323 DEBUG (OverseerStateUpdate-72104282642382857-127.0.0.1:37592_solr-n_0000000000) [n:127.0.0.1:37592_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 2386324 INFO (zkCallback-2097-thread-1-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2386324 INFO (zkCallback-2100-thread-1-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2386324 INFO (zkCallback-2095-thread-1-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2386324 INFO (zkCallback-2095-thread-2-processing-n:127.0.0.1:40206_solr) [n:127.0.0.1:40206_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2386324 INFO (zkCallback-2097-thread-2-processing-n:127.0.0.1:33018_solr) [n:127.0.0.1:33018_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2386324 INFO (zkCallback-2100-thread-2-processing-n:127.0.0.1:37592_solr) [n:127.0.0.1:37592_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2387224 INFO (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2387224 INFO (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2387224 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 2387225 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2387225 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 2387225 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2387225 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
[junit4] 2> 2387250 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2387251 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
[junit4] 2> 2387251 WARN (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2387267 INFO (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2387268 INFO (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2387268 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 2387300 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2387301 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 2387301 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2387301 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 2387301 INFO (qtp1526287941-13291) [n:127.0.0.1:33018_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2387301 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2387301 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
[junit4] 2> 2387304 WARN (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2387395 INFO (qtp62202932-13305) [n:127.0.0.1:37592_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2387395 DEBUG (ScheduledTrigger-3833-thread-2) [n:127.0.0.1:37592_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2387429 INFO (qtp1347073129-13279) [n:127.0.0.1:40206_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2387430 INFO (qtp1347073129-13279) [n:127.0.0.1:40206_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2387430 WARN (qtp1347073129-13279) [n:127.0.0.1:40206_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
[junit4] 2> 2387430 WARN (qtp1347073129-13279) [n:127.0.0.1:40206_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_23295B88B7E20539-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/
[...truncated too long message...]
[ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3903b8b6{/solr,null,UNAVAILABLE}
[junit4] 2> 4828075 INFO (jetty-closer-4391-thread-10) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4828080 INFO (jetty-closer-4391-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@14b44350{/solr,null,UNAVAILABLE}
[junit4] 2> 4828082 INFO (jetty-closer-4391-thread-6) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4828085 INFO (jetty-closer-4391-thread-9) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4eaf9891{/solr,null,UNAVAILABLE}
[junit4] 2> 4828085 INFO (jetty-closer-4391-thread-9) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4829444 WARN (zkCallback-4520-thread-1-processing-n:127.0.0.1:43629_solr) [n:127.0.0.1:43629_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4829444 WARN (zkCallback-4616-thread-1-processing-n:127.0.0.1:38937_solr) [n:127.0.0.1:38937_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4829444 INFO (jetty-closer-4391-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@418552{/solr,null,UNAVAILABLE}
[junit4] 2> 4829444 INFO (jetty-closer-4391-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4829444 INFO (jetty-closer-4391-thread-13) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4aa829de{/solr,null,UNAVAILABLE}
[junit4] 2> 4829444 INFO (jetty-closer-4391-thread-13) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4829460 INFO (jetty-closer-4391-thread-14) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@76468759{/solr,null,UNAVAILABLE}
[junit4] 2> 4829461 INFO (jetty-closer-4391-thread-14) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4829466 ERROR (SUITE-TriggerIntegrationTest-seed#[23295B88B7E20539]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4829466 INFO (SUITE-TriggerIntegrationTest-seed#[23295B88B7E20539]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44815 44815
[junit4] 2> 4829589 INFO (Thread-6602) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44815 44815
[junit4] 2> 4829593 WARN (Thread-6602) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 35 /solr/collections/collection1/terms/shard1
[junit4] 2> 23 /solr/clusterprops.json
[junit4] 2> 23 /solr/aliases.json
[junit4] 2> 22 /solr/security.json
[junit4] 2> 11 /solr/configs/conf
[junit4] 2> 9 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 110 /solr/collections/collection1/state.json
[junit4] 2> 41 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 23 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 4 /solr/overseer_elect/election/72104428282576950-127.0.0.1:38143_solr-n_0000000016
[junit4] 2> 3 /solr/overseer_elect/election/72104428282576923-127.0.0.1:38820_solr-n_0000000007
[junit4] 2> 3 /solr/overseer_elect/election/72104428282576926-127.0.0.1:45046_solr-n_0000000008
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72104428282576941-core_node3-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/72104428282576941-127.0.0.1:43604_solr-n_0000000013
[junit4] 2> 2 /solr/overseer_elect/election/72104428282576902-127.0.0.1:45756_solr-n_0000000001
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72104428282576923-core_node4-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/72104428282576938-127.0.0.1:43042_solr-n_0000000012
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/live_nodes
[junit4] 2> 23 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_23295B88B7E20539-001
[junit4] 2> Feb 11, 2018 7:30:56 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct), x_s=FSTOrd50}, docValues:{_version_=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=475, maxMBSortInHeap=7.2602295291808865, sim=RandomSimilarity(queryNorm=false): {}, locale=pl-PL, timezone=America/Sitka
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=106648472,total=478150656
[junit4] 2> NOTE: All tests run in this JVM: [UniqFieldsUpdateProcessorFactoryTest, MoveReplicaHDFSTest, SolrCmdDistributorTest, TestLeaderElectionWithEmptyReplica, DistributedFacetPivotLargeTest, TestReqParamsAPI, TestFastWriter, ResponseLogComponentTest, TestSimDistributedQueue, TestFileDictionaryLookup, PreAnalyzedUpdateProcessorTest, TestSolrCloudWithDelegationTokens, TestHashQParserPlugin, TestTrieFacet, ForceLeaderTest, LeaderElectionTest, TestCustomStream, TestDistributedStatsComponentCardinality, CustomCollectionTest, TestLazyCores, LeaderFailureAfterFreshStartTest, TestLockTree, SolrGangliaReporterTest, TestStressCloudBlindAtomicUpdates, TestHdfsUpdateLog, TestLegacyFieldCache, TestSolrCoreProperties, NodeMutatorTest, AnalyticsQueryTest, MigrateRouteKeyTest, UpdateLogTest, TestNumericRangeQuery64, SuggesterTSTTest, ClusterStateUpdateTest, TestUtils, TestMacroExpander, SearchHandlerTest, DistributedFacetPivotSmallTest, TestFieldCache, DistribCursorPagingTest, SolrMetricManagerTest, HighlighterMaxOffsetTest, SpellPossibilityIteratorTest, TestSolrDeletionPolicy2, NoCacheHeaderTest, MetricUtilsTest, TestFieldSortValues, TestRawTransformer, AssignTest, SolrIndexMetricsTest, ClusterStateTest, TestValueSourceCache, StatsReloadRaceTest, TestSmileRequest, SolrXmlInZkTest, HLLSerializationTest, TestUninvertingReader, SimpleCollectionCreateDeleteTest, TestCloudInspectUtil, TestDFRSimilarityFactory, TestClusterStateProvider, BlockCacheTest, TestLFUCache, DocValuesTest, TestInfoStreamLogging, RollingRestartTest, LIRRollingUpdatesTest, TestNamedUpdateProcessors, TestSubQueryTransformerDistrib, TestSchemaManager, TestNodeAddedTrigger, AtomicUpdateProcessorFactoryTest, TestConfigSetProperties, SharedFSAutoReplicaFailoverTest, TestSolrCLIRunExample, TestDistribDocBasedVersion, CdcrUpdateLogTest, TestAnalyzedSuggestions, HttpSolrCallGetCoreTest, DocExpirationUpdateProcessorFactoryTest, TestDistributedMissingSort, CacheHeaderTest, PKIAuthenticationIntegrationTest, TestQueryTypes, TestFiltering, ClassificationUpdateProcessorIntegrationTest, TestLMJelinekMercerSimilarityFactory, TestAtomicUpdateErrorCases, OverseerCollectionConfigSetProcessorTest, TestSQLHandlerNonCloud, TestComplexPhraseLeadingWildcard, TestCoreDiscovery, TestQuerySenderListener, DeleteReplicaTest, JvmMetricsTest, TestJsonRequest, AutoscalingHistoryHandlerTest, TestTolerantSearch, TestReloadAndDeleteDocs, TestFieldResource, TestRuleBasedAuthorizationPlugin, TestChildDocTransformer, ConnectionManagerTest, TestPayloadScoreQParserPlugin, NumericFieldsTest, LukeRequestHandlerTest, SpellCheckCollatorWithCollapseTest, SearchHandlerTest, TestFaceting, CdcrReplicationHandlerTest, DistributedQueryComponentOptimizationTest, TestCodecSupport, TestRealTimeGet, CoreMergeIndexesAdminHandlerTest, TestLegacyField, DistributedDebugComponentTest, TestConfigsApi, TestLRUStatsCache, TestClassicSimilarityFactory, DistributedSpellCheckComponentTest, ConvertedLegacyTest, TestJavabinTupleStreamParser, TestPrepRecovery, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, RecoveryZkTest, TestZkChroot, TestRandomDVFaceting, TestStressReorder, SpellCheckComponentTest, TestGroupingSearch, BadIndexSchemaTest, TestFunctionQuery, SoftAutoCommitTest, PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, DateFieldTest, SystemInfoHandlerTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, ActionThrottleTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteInactiveReplicaTest, DistributedQueueTest, DocValuesNotIndexedTest, LegacyCloudClusterPropTest, MultiThreadedOCPTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, TestAuthenticationFramework, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestPullReplicaErrorHandling, TestShortCircuitedRequests, TestStressInPlaceUpdates, TestTolerantUpdateProcessorRandomCloud, VMParamsZkACLAndCredentialsProvidersTest, CollectionReloadTest, CollectionsAPIAsyncDistributedZkTest, TestCollectionAPI, TestCollectionsAPIViaSolrCloudCluster, TestLocalFSCloudBackupRestore, TestRequestStatusCollectionAPI, ComputePlanActionTest, HdfsAutoAddReplicasIntegrationTest, TriggerIntegrationTest]
[junit4] Completed [622/775 (3!)] on J2 in 224.34s, 14 tests, 1 failure <<< FAILURES!
[...truncated 48903 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 371 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/371/
3 tests failed.
FAILED: org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:44658/solr/testNewReplicaOldLeader
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:44658/solr/testNewReplicaOldLeader
at __randomizedtesting.SeedInfo.seed([7E003417CD34A441:C3569EC92DEADA9F]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.LIRRollingUpdatesTest.realTimeGetDocId(LIRRollingUpdatesTest.java:393)
at org.apache.solr.cloud.LIRRollingUpdatesTest.assertDocExists(LIRRollingUpdatesTest.java:385)
at org.apache.solr.cloud.LIRRollingUpdatesTest.assertDocsExistInAllReplicas(LIRRollingUpdatesTest.java:369)
at org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader(LIRRollingUpdatesTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 46 more
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.preferLocalShardsTest
Error Message:
Could not load collection from ZK: localShardsTestColl
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: localShardsTestColl
at __randomizedtesting.SeedInfo.seed([5DBF218E63CFBE8D:A172B9379D182977]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1206)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:849)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithPreferLocalShards(CloudSolrClientTest.java:435)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.preferLocalShardsTest(CloudSolrClientTest.java:416)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/localShardsTestColl/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 48 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([7E003417CD34A441:23482A9E02F2020E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12707 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_7E003417CD34A441-001/init-core-data-001
[junit4] 2> 1406775 INFO (SUITE-TestTriggerIntegration-seed#[7E003417CD34A441]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1406776 INFO (SUITE-TestTriggerIntegration-seed#[7E003417CD34A441]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1406776 INFO (SUITE-TestTriggerIntegration-seed#[7E003417CD34A441]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1406794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1406794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1406794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
[junit4] 2> 1406794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1406794 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1406795 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1406795 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1406795 DEBUG (ScheduledTrigger-3125-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1406803 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 1406803 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1406803 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1406803 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1406803 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1406822 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1406822 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1406823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1406823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1406823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1406823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1406823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1406823 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1406843 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1406888 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1406911 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1406917 DEBUG (simCloudManagerPool-3127-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1406917 DEBUG (simCloudManagerPool-3127-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1407060 DEBUG (simCloudManagerPool-3127-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407060 DEBUG (simCloudManagerPool-3127-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407060 DEBUG (simCloudManagerPool-3127-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1407060 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1407060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1407119 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1407119 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1407119 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1407142 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407156 DEBUG (simCloudManagerPool-3127-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407157 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1407157 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1407157 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1407157 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1407157 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1407176 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1407176 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1407176 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10018_solr at time 737693304944149
[junit4] 2> 1407177 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10018_solr] added at times [737693304944149], now=737693306776099
[junit4] 2> 1407177 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29eed9b64f615T2ghu285f1fqs7uusq0ctkw66k",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":737693304944149,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737693304944149],
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407177 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 1407240 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29eed9b64f615T2ghu285f1fqs7uusq0ctkw66k",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":737693304944149,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737693304944149],
[junit4] 2> "_enqueue_time_":737693322439649,
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407240 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 1407314 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1407314 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 1407314 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 1407314 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 3712 ms for event id=29eed9b64f615T2ghu285f1fqs7uusq0ctkw66k
[junit4] 2> 1407416 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10018_solr at time 737712042971799
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10018_solr] added at times [737712042971799], now=737712045045299
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29ef1f8449297T2ghu285f1fqs7uusq0ctkw66n",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":737712042971799,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737712042971799],
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407551 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 1407587 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29ef1f8449297T2ghu285f1fqs7uusq0ctkw66n",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":737712042971799,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737712042971799],
[junit4] 2> "_enqueue_time_":737712055783399,
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407587 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 737699193949299 time = 737713836879599
[junit4] 2> 1407587 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 1407589 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1407589 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 1407589 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 1407589 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 113 ms for event id=29ef1f8449297T2ghu285f1fqs7uusq0ctkw66n
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407610 DEBUG (simCloudManagerPool-3127-thread-7) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407624 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1407624 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1407624 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1407632 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 1407652 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 1407652 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1407652 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1407652 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407652 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407652 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407653 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407653 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1407653 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407653 DEBUG (simCloudManagerPool-3127-thread-8) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1407653 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1407653 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1407653 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1407653 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
[junit4] 2> 1407737 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 1407758 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10018_solr
[junit4] 2> 1407770 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10018_solr]
[junit4] 2> 1407770 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29ef484ec419dT2ghu285f1fqs7uusq0ctkw66q",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":737722992705949,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737722992705949],
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407770 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 1407741 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1407741 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10018_solr
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10018_solr
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10018_solr]
[junit4] 2> 1407771 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29ef48761765fT2ghu285f1fqs7uusq0ctkw66s",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":737723033941599,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737723033941599],
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407786 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29ef484ec419dT2ghu285f1fqs7uusq0ctkw66q",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":737722992705949,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737722992705949],
[junit4] 2> "_enqueue_time_":737723005075299,
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407786 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 1407790 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 204 ms for event id=29ef484ec419dT2ghu285f1fqs7uusq0ctkw66q
[junit4] 2> 1407815 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger processor for lost nodes: [127.0.0.1:10018_solr] is not ready, will try later
[junit4] 2> 1407892 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1407892 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1407892 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1407892 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1407893 DEBUG (ScheduledTrigger-3128-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1407912 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 1407912 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10018_solr]
[junit4] 2> 1407912 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29ef48761765fT2ghu285f1fqs7uusq0ctkw66v",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":737723033941599,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737723033941599],
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407913 DEBUG (ScheduledTrigger-3128-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 1407913 DEBUG (ScheduledTrigger-3128-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 1407935 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29ef48761765fT2ghu285f1fqs7uusq0ctkw66v",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":737723033941599,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737723033941599],
[junit4] 2> "_enqueue_time_":737730109898099,
[junit4] 2> "nodeNames":["127.0.0.1:10018_solr"]}}
[junit4] 2> 1407935 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 737723772389649 time = 737731233389449
[junit4] 2> 1407935 INFO (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10018_solr
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1407935 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 4
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 8
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger2
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger2
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - IGNORED 1
[junit4] 2> 1407936 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1408072 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 1408072 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1408072 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 1408073 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 1408073 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 1408073 DEBUG (AutoscalingActionExecutor-3129-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 6907 ms for event id=29ef48761765fT2ghu285f1fqs7uusq0ctkw66v
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1408250 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1408275 DEBUG (ScheduledTrigger-3128-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1408276 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
[junit4] 2> 1408276 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1408276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1408276 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10018_solr
[junit4] 2> 1408276 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1408276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1408276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1408276 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1408276 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1408278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1408278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1408278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1408278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion -1
[junit4] 2> 1408278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1408291 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1408291 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1408295 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1408315 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1408396 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1408418 DEBUG (simCloudManagerPool-3130-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1408418 DEBUG (simCloudManagerPool-3130-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1408418 DEBUG (simCloudManagerPool-3130-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1408418 DEBUG (simCloudManagerPool-3130-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1408418 DEBUG (simCloudManagerPool-3130-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1408419 DEBUG (simCloudManagerPool-3130-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1408420 DEBUG (simCloudManagerPool-3130-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1408420 DEBUG (simCloudManagerPool-3130-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1408420 DEBUG (simCloudManagerPool-3130-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1408420 DEBUG (simCloudManagerPool-3130-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1408429 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1408429 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1408449 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1408534 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 1408535 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1408535 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10019_solr at time 737761210512249
[junit4] 2> 1408607 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 1408607 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1408608 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10019_solr] added at times [737761210512249], now=737764858094049
[junit4] 2> 1408608 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29efd6ae1b379T2ghu285f1fqs7uusq0ctkw66y",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":737761210512249,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737761210512249],
[junit4] 2> "nodeNames":["127.0.0.1:10019_solr"]}}
[junit4] 2> 1408610 DEBUG (ScheduledTrigger-3131-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1408676 DEBUG (ScheduledTrigger-3131-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
[junit4] 2> 1408677 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29efd6ae1b379T2ghu285f1fqs7uusq0ctkw66y",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":737761210512249,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737761210512249],
[junit4] 2> "_enqueue_time_":737768301233599,
[junit4] 2> "nodeNames":["127.0.0.1:10019_solr"]}}
[junit4] 2> 1408679 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1408679 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
[junit4] 2> 1408679 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 96 ms for event id=29efd6ae1b379T2ghu285f1fqs7uusq0ctkw66y
[junit4] 2> 1408782 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 1408863 DEBUG (ScheduledTrigger-3131-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 1408875 DEBUG (ScheduledTrigger-3131-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1408875 DEBUG (ScheduledTrigger-3131-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10020_solr at time 737778237889699
[junit4] 2> 1408899 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 1408899 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 1408899 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1408900 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10020_solr] added at times [737778237889699], now=737779455366199
[junit4] 2> 1408900 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29f0161cadca3T2ghu285f1fqs7uusq0ctkw671",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":737778237889699,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737778237889699],
[junit4] 2> "nodeNames":["127.0.0.1:10020_solr"]}}
[junit4] 2> 1408900 DEBUG (ScheduledTrigger-3131-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
[junit4] 2> 1408936 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29f0161cadca3T2ghu285f1fqs7uusq0ctkw671",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":737778237889699,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737778237889699],
[junit4] 2> "_enqueue_time_":737779465698399,
[junit4] 2> "nodeNames":["127.0.0.1:10020_solr"]}}
[junit4] 2> 1408972 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1408973 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
[junit4] 2> 1408973 DEBUG (AutoscalingActionExecutor-3132-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1829 ms for event id=29f0161cadca3T2ghu285f1fqs7uusq0ctkw671
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCooldown
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 4
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 4
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 4
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_cooldown_trigger
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 1408993 INFO (TEST-TestTriggerIntegration.testCooldown-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 2
[junit4] 2> 1409011 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 1409011 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1409012 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1409012 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1409021 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1409029 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1409029 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 1409029 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409029 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion -1
[junit4] 2> 1409029 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 1409030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1409030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 1409030 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409066 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409103 DEBUG (ScheduledTrigger-3134-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409123 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409143 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409144 DEBUG (simCloudManagerPool-3133-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1409144 DEBUG (simCloudManagerPool-3133-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 1409144 DEBUG (simCloudManagerPool-3133-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409144 DEBUG (simCloudManagerPool-3133-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1409144 DEBUG (simCloudManagerPool-3133-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409144 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 1409144 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1409144 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 1409145 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1409145 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1409180 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409180 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1409180 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409180 DEBUG (ScheduledTrigger-3134-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10021_solr at time 737793488002649
[junit4] 2> 1409210 DEBUG (simCloudManagerPool-3133-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1409210 DEBUG (simCloudManagerPool-3133-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 1409210 DEBUG (simCloudManagerPool-3133-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409210 DEBUG (simCloudManagerPool-3133-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1409210 DEBUG (simCloudManagerPool-3133-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 1409210 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1409210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10021_solr
[junit4] 2> 1409211 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 1409218 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409218 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1409218 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409218 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10021_solr] added at times [737795004492749], now=737795391253549
[junit4] 2> 1409218 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29f054928ebcdT2ghu285f1fqs7uusq0ctkw674",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":737795004492749,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737795004492749],
[junit4] 2> "nodeNames":["127.0.0.1:10021_solr"]}}
[junit4] 2> 1409233 DEBUG (ScheduledTrigger-3134-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 1409270 DEBUG (AutoscalingActionExecutor-3135-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"29f054928ebcdT2ghu285f1fqs7uusq0ctkw674",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":737795004492749,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737795004492749],
[junit4] 2> "_enqueue_time_":737796119558999,
[junit4] 2> "nodeNames":["127.0.0.1:10021_solr"]}}
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 1409271 DEBUG (AutoscalingActionExecutor-3135-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 1409271 DEBUG (AutoscalingActionExecutor-3135-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 1409271 DEBUG (AutoscalingActionExecutor-3135-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 43 ms for event id=29f054928ebcdT2ghu285f1fqs7uusq0ctkw674
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_restore_trigger
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1409271 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1409320 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7E003417CD34A441]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventFromRestoredState
[junit4] 2> 1409320 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 1409320 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1409320 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1409321 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7E003417CD34A441]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1409355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1409355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 1409355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion -1
[junit4] 2> 1409355 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 1409356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1409356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 1409356 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409387 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409473 DEBUG (ScheduledTrigger-3137-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409493 DEBUG (ScheduledTrigger-3137-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1409496 DEBUG (simCloudManagerPool-3136-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1409497 DEBUG (simCloudManagerPool-3136-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 1409497 DEBUG (simCloudManagerPool-3136-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409497 DEBUG (simCloudManagerPool-3136-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
[junit4] 2> 1409497 DEBUG (simCloudManagerPool-3136-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1409497 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 1409497 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1409498 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 1409498 DEBUG (ScheduledTrigger-3137-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409498 DEBUG (ScheduledTrigger-3137-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409498 DEBUG (ScheduledTrigger-3137-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10022_solr at time 737809366062149
[junit4] 2> 1409525 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409525 DEBUG (ScheduledTrigger-3137-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409526 DEBUG (ScheduledTrigger-3137-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409558 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409558 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409558 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409594 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409594 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409594 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409614 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409614 DEBUG (ScheduledTrigger-3137-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409614 DEBUG (ScheduledTrigger-3137-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409635 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409635 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409635 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409655 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409655 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409655 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409676 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409676 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409676 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409700 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1409700 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1409700 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1409700 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10022_solr] added at times [737809366062149], now=737819496193349
[junit4] 2> 1409700 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"29f08a12d1445T2ghu285f1fqs7uusq0ctkw677",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":737809366062149,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[737809366062149],
[junit4] 2> "nodeNames":["127.0.0.1:10022_solr"]}}
[junit4] 2> 1409701 DEBUG (ScheduledTrigger-3137-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 1409712 DEBUG (Autoscaling
[...truncated too long message...]
2_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=1226546304
[junit4] 2> 175983 INFO (coreCloseExecutor-355-thread-1) [n:127.0.0.1:42124_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader, tag=651717002
[junit4] 2> 176004 INFO (coreCloseExecutor-356-thread-6) [n:127.0.0.1:45135_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n12] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=2102503131
[junit4] 2> 176004 INFO (coreCloseExecutor-356-thread-8) [n:127.0.0.1:45135_solr c:2nd_collection s:shard1 r:core_node3 x:2nd_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.2nd_collection.shard1.replica_n1, tag=1949393332
[junit4] 2> 176005 INFO (coreCloseExecutor-356-thread-8) [n:127.0.0.1:45135_solr c:2nd_collection s:shard1 r:core_node3 x:2nd_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@79c443ab: rootName = solr_45135, domain = solr.core.2nd_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.2nd_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7aea3db5
[junit4] 2> 176399 INFO (coreCloseExecutor-354-thread-6) [n:127.0.0.1:45303_solr c:localShardsTestColl s:shard1 r:core_node5 x:localShardsTestColl_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=140924220
[junit4] 2> 176400 INFO (coreCloseExecutor-354-thread-2) [n:127.0.0.1:45303_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.foo.shard1.replica_n4, tag=1259896135
[junit4] 2> 176400 INFO (coreCloseExecutor-354-thread-2) [n:127.0.0.1:45303_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@22168a05: rootName = solr_45303, domain = solr.core.foo.shard1.replica_n4, service url = null, agent id = null] for registry solr.core.foo.shard1.replica_n4 / com.codahale.metrics.MetricRegistry@c093f26
[junit4] 2> 176425 INFO (coreCloseExecutor-356-thread-8) [n:127.0.0.1:45135_solr c:2nd_collection s:shard1 r:core_node3 x:2nd_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard1.leader, tag=1949393332
[junit4] 2> 176450 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10029e964f40008, likely client has closed socket
[junit4] 2> 176454 INFO (jetty-closer-874-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6b90df0d{/solr,null,UNAVAILABLE}
[junit4] 2> 176454 INFO (jetty-closer-874-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 176470 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10029e964f40009, likely client has closed socket
[junit4] 2> 176475 INFO (coreCloseExecutor-354-thread-2) [n:127.0.0.1:45303_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=1259896135
[junit4] 2> 176475 INFO (coreCloseExecutor-354-thread-8) [n:127.0.0.1:45303_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader, tag=2054984331
[junit4] 2> 176500 INFO (jetty-closer-874-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72103676435759111-127.0.0.1:45303_solr-n_0000000000) closing
[junit4] 2> 176500 INFO (OverseerStateUpdate-72103676435759111-127.0.0.1:45303_solr-n_0000000000) [n:127.0.0.1:45303_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45303_solr
[junit4] 2> 176508 WARN (OverseerAutoScalingTriggerThread-72103676435759111-127.0.0.1:45303_solr-n_0000000000) [n:127.0.0.1:45303_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 176512 INFO (jetty-closer-874-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@49c822da{/solr,null,UNAVAILABLE}
[junit4] 2> 176512 INFO (jetty-closer-874-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 177955 INFO (jetty-closer-874-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1227fd37{/solr,null,UNAVAILABLE}
[junit4] 2> 177956 INFO (jetty-closer-874-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 177957 ERROR (SUITE-CloudSolrClientTest-seed#[5DBF218E63CFBE8D]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 177957 INFO (SUITE-CloudSolrClientTest-seed#[5DBF218E63CFBE8D]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46339 46339
[junit4] 2> 178237 INFO (Thread-136) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46339 46339
[junit4] 2> 178238 WARN (Thread-136) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/collections/foo/terms/shard1
[junit4] 2> 9 /solr/aliases.json
[junit4] 2> 9 /solr/collections/localShardsTestColl/terms/shard2
[junit4] 2> 9 /solr/collections/localShardsTestColl/terms/shard1
[junit4] 2> 9 /solr/collections/localShardsTestColl/terms/shard3
[junit4] 2> 6 /solr/collections/stale_state_test_col/terms/shard1
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2> 2 /solr/collections/overwrite/terms/shard1
[junit4] 2> 2 /solr/collections/multicollection1/terms/shard2
[junit4] 2> 2 /solr/collections/multicollection1/terms/shard1
[junit4] 2> 2 /solr/collections/2nd_collection/terms/shard2
[junit4] 2> 2 /solr/collections/collection1/terms/shard1
[junit4] 2> 2 /solr/collections/2nd_collection/terms/shard1
[junit4] 2> 2 /solr/collections/collection1/terms/shard2
[junit4] 2> 2 /solr/collections/multicollection2/terms/shard2
[junit4] 2> 2 /solr/collections/multicollection2/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 39 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 24 /solr/collections/foo/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 16 /solr/collections/2nd_collection/state.json
[junit4] 2> 16 /solr/collections/multicollection2/state.json
[junit4] 2> 14 /solr/collections/collection1/state.json
[junit4] 2> 12 /solr/collections/multicollection1/state.json
[junit4] 2> 9 /solr/clusterprops.json
[junit4] 2> 9 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/live_nodes
[junit4] 2> 9 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.CloudSolrClientTest_5DBF218E63CFBE8D-001
[junit4] 2> Feb 11, 2018 4:17:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=Lucene50(blocksize=128), a_t=Lucene50(blocksize=128), title_s=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapFixedInterval), text=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Direct), multiDefault=DocValuesFormat(name=Lucene70), title_s=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=493, maxMBSortInHeap=6.188897859862683, sim=RandomSimilarity(queryNorm=false): {}, locale=lt-LT, timezone=America/Montreal
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=275400000,total=444071936
[junit4] 2> NOTE: All tests run in this JVM: [TestV1toV2ApiMapper, FieldValueEvaluatorTest, TestConfigSetAdminRequest, TestSpellCheckResponse, LessThanEvaluatorTest, TermsResponseTest, TestZkConfigManager, AscEvaluatorTest, TestEmbeddedSolrServer, NotEvaluatorTest, TestToleratedUpdateError, TestPathTrie, JdbcTest, TestXMLEscaping, EmpiricalDistributionEvaluatorTest, ConcurrentUpdateSolrClientBuilderTest, TestPolicy, CloudSolrClientCacheTest, CubedRootEvaluatorTest, GraphExpressionTest, ContentStreamTest, ReverseEvaluatorTest, SolrExampleStreamingTest, SolrExampleBinaryTest, LargeVolumeBinaryJettyTest, TestBatchUpdate, LargeVolumeEmbeddedTest, SolrParamTest, TestNamedListCodec, TestDocumentObjectBinder, SolrQueryTest, DocumentAnalysisResponseTest, SolrExceptionTest, FieldAnalysisResponseTest, AnlysisResponseBaseTest, UsingSolrJRefGuideExamplesTest, CloudSolrClientMultiConstructorTest, CloudSolrClientTest]
[junit4] Completed [126/146 (1!)] on J0 in 67.67s, 15 tests, 1 error <<< FAILURES!
[...truncated 47493 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 370 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/370/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSkipOverseerOperations
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSkipOverseerOperations: 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSkipOverseerOperations:
1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([E0A318B8151D8406]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSkipOverseerOperations
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([E0A318B8151D8406]:0)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([E0A318B8151D8406:68F72762BBE1E9FE]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13806 lines...]
[junit4] Suite: org.apache.solr.cloud.ReplaceNodeNoTargetTest
[junit4] 2> 1935227 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/init-core-data-001
[junit4] 2> 1935227 WARN (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 1935228 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1935229 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1935230 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001
[junit4] 2> 1935230 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1935230 INFO (Thread-4912) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1935230 INFO (Thread-4912) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1935233 ERROR (Thread-4912) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1935330 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:48757
[junit4] 2> 1935333 INFO (zkConnectionManagerCallback-3443-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935337 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935337 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935337 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935338 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935338 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935338 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935338 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1935338 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4856d69b{/solr,null,AVAILABLE}
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@330bb9af{HTTP/1.1,[http/1.1]}{127.0.0.1:48164}
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.e.j.s.Server Started @1937381ms
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48164}
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1935339 ERROR (jetty-launcher-3440-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70c3b433{/solr,null,AVAILABLE}
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.824Z
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31f2c9d2{HTTP/1.1,[http/1.1]}{127.0.0.1:48451}
[junit4] 2> 1935339 INFO (jetty-launcher-3440-thread-1) [ ] o.e.j.s.Server Started @1937381ms
[junit4] 2> 1935340 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48451}
[junit4] 2> 1935340 ERROR (jetty-launcher-3440-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935340 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935340 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935340 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935340 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.825Z
[junit4] 2> 1935341 INFO (zkConnectionManagerCallback-3445-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935342 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935348 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935349 INFO (zkConnectionManagerCallback-3447-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935349 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23c37fe8{/solr,null,AVAILABLE}
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e21fae{/solr,null,AVAILABLE}
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49c7c8f2{HTTP/1.1,[http/1.1]}{127.0.0.1:36880}
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.e.j.s.Server Started @1937392ms
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36880}
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1a286baa{HTTP/1.1,[http/1.1]}{127.0.0.1:56832}
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.e.j.s.Server Started @1937392ms
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56832}
[junit4] 2> 1935350 ERROR (jetty-launcher-3440-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935350 ERROR (jetty-launcher-3440-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935350 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935351 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935351 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.836Z
[junit4] 2> 1935351 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.836Z
[junit4] 2> 1935351 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935353 INFO (jetty-launcher-3440-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935353 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935353 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@256a8f0e{/solr,null,AVAILABLE}
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7868b76e{HTTP/1.1,[http/1.1]}{127.0.0.1:35902}
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.e.j.s.Server Started @1937396ms
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35902}
[junit4] 2> 1935354 ERROR (jetty-launcher-3440-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935354 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935355 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935355 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935355 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.840Z
[junit4] 2> 1935355 INFO (zkConnectionManagerCallback-3451-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935356 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10454e0619c0002, likely client has closed socket
[junit4] 2> 1935357 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935358 INFO (zkConnectionManagerCallback-3455-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935360 INFO (zkConnectionManagerCallback-3450-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935360 INFO (jetty-launcher-3440-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935360 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935361 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T07:57:37+10:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1935363 INFO (zkConnectionManagerCallback-3457-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935364 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935366 INFO (jetty-launcher-3440-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935368 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1935368 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1935368 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1935369 INFO (zkConnectionManagerCallback-3465-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935369 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30723646{/solr,null,AVAILABLE}
[junit4] 2> 1935369 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@34e5d6e5{HTTP/1.1,[http/1.1]}{127.0.0.1:33319}
[junit4] 2> 1935369 INFO (jetty-launcher-3440-thread-6) [ ] o.e.j.s.Server Started @1937411ms
[junit4] 2> 1935369 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33319}
[junit4] 2> 1935369 ERROR (jetty-launcher-3440-thread-6) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1935370 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1935370 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1935370 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1935370 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T12:25:49.855Z
[junit4] 2> 1935370 INFO (jetty-launcher-3440-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935371 INFO (zkConnectionManagerCallback-3467-thread-1-processing-n:127.0.0.1:48451_solr) [n:127.0.0.1:48451_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935371 INFO (zkConnectionManagerCallback-3461-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935372 INFO (zkConnectionManagerCallback-3471-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935373 INFO (zkConnectionManagerCallback-3473-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935373 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1935374 INFO (jetty-launcher-3440-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935375 INFO (zkConnectionManagerCallback-3475-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935375 INFO (zkConnectionManagerCallback-3478-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935405 INFO (zkConnectionManagerCallback-3481-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935406 INFO (zkConnectionManagerCallback-3483-thread-1-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935407 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10454e0619c000e, likely client has closed socket
[junit4] 2> 1935410 INFO (zkConnectionManagerCallback-3485-thread-1-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935414 INFO (jetty-launcher-3440-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48757/solr
[junit4] 2> 1935420 INFO (zkConnectionManagerCallback-3489-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935422 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10454e0619c0011, likely client has closed socket
[junit4] 2> 1935425 INFO (zkConnectionManagerCallback-3496-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935447 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935449 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33319_solr
[junit4] 2> 1935450 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.c.Overseer Overseer (id=73276816631791634-127.0.0.1:33319_solr-n_0000000000) starting
[junit4] 2> 1935462 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33319_solr
[junit4] 2> 1935480 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1935516 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1935517 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1935519 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1935519 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935519 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1935519 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935520 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935521 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35902_solr
[junit4] 2> 1935521 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36880_solr
[junit4] 2> 1935528 INFO (zkCallback-3482-thread-1-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1935528 INFO (zkCallback-3474-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1935532 INFO (zkCallback-3495-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1935536 INFO (zkCallback-3484-thread-1-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1935544 INFO (zkCallback-3477-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1935544 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56832_solr
[junit4] 2> 1935548 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935548 INFO (zkCallback-3495-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1935549 INFO (zkCallback-3474-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1935549 INFO (zkCallback-3477-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1935550 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48164_solr
[junit4] 2> 1935550 INFO (zkCallback-3477-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1935551 INFO (zkCallback-3495-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5)
[junit4] 2> 1935551 INFO (zkCallback-3484-thread-1-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1935556 INFO (zkCallback-3474-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1935557 INFO (zkCallback-3482-thread-1-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1935560 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1935560 INFO (zkCallback-3482-thread-2-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1935560 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1935561 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48451_solr
[junit4] 2> 1935562 INFO (zkCallback-3477-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 1935562 INFO (zkCallback-3482-thread-2-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 1935564 INFO (zkCallback-3474-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 1935564 INFO (zkCallback-3495-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 1935580 INFO (zkCallback-3484-thread-2-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 1935588 INFO (zkCallback-3466-thread-1-processing-n:127.0.0.1:48451_solr) [n:127.0.0.1:48451_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 1935628 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35902.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935634 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35902.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935635 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35902.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935636 INFO (jetty-launcher-3440-thread-5) [n:127.0.0.1:35902_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node5/.
[junit4] 2> 1935637 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56832.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935639 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33319.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935645 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56832.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935645 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56832.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935646 INFO (jetty-launcher-3440-thread-3) [n:127.0.0.1:56832_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node3/.
[junit4] 2> 1935674 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33319.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935674 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33319.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935675 INFO (jetty-launcher-3440-thread-6) [n:127.0.0.1:33319_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node6/.
[junit4] 2> 1935679 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36880.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935685 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48164.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935688 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36880.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935688 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36880.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935689 INFO (jetty-launcher-3440-thread-4) [n:127.0.0.1:36880_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node4/.
[junit4] 2> 1935694 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48164.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935695 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48164.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935696 INFO (jetty-launcher-3440-thread-2) [n:127.0.0.1:48164_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node2/.
[junit4] 2> 1935698 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48451.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935705 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48451.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935705 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48451.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1935706 INFO (jetty-launcher-3440-thread-1) [n:127.0.0.1:48451_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node1/.
[junit4] 2> 1935947 INFO (zkConnectionManagerCallback-3505-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935950 INFO (zkConnectionManagerCallback-3509-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935951 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 1935952 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48757/solr ready
[junit4] 2> 1935994 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[E0A318B8151D8406]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 1935995 INFO (zkConnectionManagerCallback-3511-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1935996 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[E0A318B8151D8406]) [ ] o.a.s.c.ReplaceNodeNoTargetTest total_jettys: 6
[junit4] 2> 1935999 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936000 INFO (qtp1085953888-19220) [n:127.0.0.1:35902_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936001 INFO (qtp591617657-19178) [n:127.0.0.1:36880_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936002 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936003 INFO (qtp798278088-19173) [n:127.0.0.1:48451_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936005 INFO (qtp1030597602-19191) [n:127.0.0.1:56832_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936005 INFO (qtp964672269-19165) [n:127.0.0.1:48164_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 1936009 INFO (qtp1085953888-19219) [n:127.0.0.1:35902_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&collection.configName=conf1&name=replacenodetest_coll_notarget&nrtReplicas=2&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1936011 INFO (OverseerThreadFactory-7010-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll_notarget
[junit4] 2> 1936017 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936018 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936019 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936020 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936021 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936022 INFO (qtp1030597602-19191) [n:127.0.0.1:56832_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1936032 INFO (OverseerThreadFactory-7010-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18603861180 sessionWrapper.createTime 18603861170856149, this.sessionWrapper.createTime 18603861170856149
[junit4] 2> 1936139 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56832/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936141 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48451/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936142 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48164/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936144 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36880/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936146 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35902/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936147 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33319/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936149 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33319/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936151 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35902/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936153 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36880/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936154 INFO (OverseerStateUpdate-73276816631791634-127.0.0.1:33319_solr-n_0000000000) [n:127.0.0.1:33319_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48164/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1936357 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n4&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936357 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936357 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936358 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n12&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936358 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n6&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936358 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936358 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936358 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936358 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936358 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936359 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n8&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936359 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1936360 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n10&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936360 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n16&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936362 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node20&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n18&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936364 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node17&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n14&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1936467 INFO (zkCallback-3466-thread-1-processing-n:127.0.0.1:48451_solr) [n:127.0.0.1:48451_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3495-thread-1-processing-n:127.0.0.1:33319_solr) [n:127.0.0.1:33319_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3484-thread-2-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3477-thread-1-processing-n:127.0.0.1:48164_solr) [n:127.0.0.1:48164_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936468 INFO (zkCallback-3466-thread-2-processing-n:127.0.0.1:48451_solr) [n:127.0.0.1:48451_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3484-thread-1-processing-n:127.0.0.1:35902_solr) [n:127.0.0.1:35902_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3474-thread-1-processing-n:127.0.0.1:36880_solr) [n:127.0.0.1:36880_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3482-thread-1-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1936467 INFO (zkCallback-3482-thread-2-processing-n:127.0.0.1:56832_solr) [n:127.0.0.1:56832_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 1938040 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938042 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938053 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n16] Schema name=cloud-dynamic
[junit4] 2> 1938054 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938057 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938058 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938072 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938074 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938075 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n6] Schema name=cloud-dynamic
[junit4] 2> 1938078 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938078 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938080 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1938093 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n10] Schema name=cloud-dynamic
[junit4] 2> 1938097 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n4] Schema name=cloud-dynamic
[junit4] 2> 1938097 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n2] Schema name=cloud-dynamic
[junit4] 2> 1938098 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n1] Schema name=cloud-dynamic
[junit4] 2> 1938111 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n12] Schema name=cloud-dynamic
[junit4] 2> 1938118 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n14] Schema name=cloud-dynamic
[junit4] 2> 1938118 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n18] Schema name=cloud-dynamic
[junit4] 2> 1938121 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938121 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n16' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938121 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n8] Schema name=cloud-dynamic
[junit4] 2> 1938122 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36880.solr.core.replacenodetest_coll_notarget.shard5.replica_n16' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n16') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938122 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938122 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n16] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node4/replacenodetest_coll_notarget_shard5_replica_n16], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard5_replica_n16/data/]
[junit4] 2> 1938151 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938151 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n6' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938152 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36880.solr.core.replacenodetest_coll_notarget.shard2.replica_n6' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938152 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938152 INFO (qtp591617657-19196) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node4/replacenodetest_coll_notarget_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard2_replica_n6/data/]
[junit4] 2> 1938158 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938158 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n8' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938158 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35902.solr.core.replacenodetest_coll_notarget.shard3.replica_n8' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938158 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938159 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n8] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node5/replacenodetest_coll_notarget_shard3_replica_n8], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node5/./replacenodetest_coll_notarget_shard3_replica_n8/data/]
[junit4] 2> 1938172 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938172 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n4' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938176 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938176 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n2' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938176 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48451.solr.core.replacenodetest_coll_notarget.shard1.replica_n2' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938176 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938176 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node1/replacenodetest_coll_notarget_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node1/./replacenodetest_coll_notarget_shard1_replica_n2/data/]
[junit4] 2> 1938181 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938181 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n10' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938181 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33319.solr.core.replacenodetest_coll_notarget.shard3.replica_n10' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938181 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938181 INFO (qtp802920488-19239) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n10] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node6/replacenodetest_coll_notarget_shard3_replica_n10], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node6/./replacenodetest_coll_notarget_shard3_replica_n10/data/]
[junit4] 2> 1938183 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48164.solr.core.replacenodetest_coll_notarget.shard2.replica_n4' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938183 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938183 INFO (qtp964672269-19164) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node2/replacenodetest_coll_notarget_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard2_replica_n4/data/]
[junit4] 2> 1938195 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938195 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n18' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938195 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48164.solr.core.replacenodetest_coll_notarget.shard5.replica_n18' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n18') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938195 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938195 INFO (qtp964672269-19169) [n:127.0.0.1:48164_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n18] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node2/replacenodetest_coll_notarget_shard5_replica_n18], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard5_replica_n18/data/]
[junit4] 2> 1938211 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938211 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n14' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938219 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35902.solr.core.replacenodetest_coll_notarget.shard4.replica_n14' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938219 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938217 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1938219 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n1' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938220 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56832.solr.core.replacenodetest_coll_notarget.shard1.replica_n1' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938220 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938220 INFO (qtp1030597602-19203) [n:127.0.0.1:56832_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node3/replacenodetest_coll_notarget_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node3/./replacenodetest_coll_notarget_shard1_replica_n1/data/]
[junit4] 2> 1938221 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n12' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 1938222 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33319.solr.core.replacenodetest_coll_notarget.shard4.replica_n12' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e061eaf
[junit4] 2> 1938222 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938222 INFO (qtp802920488-19244) [n:127.0.0.1:33319_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n12] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node6/replacenodetest_coll_notarget_shard4_replica_n12], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node6/./replacenodetest_coll_notarget_shard4_replica_n12/data/]
[junit4] 2> 1938223 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1938223 INFO (qtp1085953888-19217) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n14] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node5/replacenodetest_coll_notarget_shard4_replica_n14], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_E0A318B8151D8406-001/tempDir-001/node5/./replacenodetest_coll_notarget_shard4_replica_n14/data/]
[junit4] 2> 1938338 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1938338 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1938343 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1938343 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1938354 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.SolrIndexSearcher Opening [Searcher@7ad5a633[replacenodetest_coll_notarget_shard5_replica_n16] main]
[junit4] 2> 1938355 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1938358 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1938376 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1938376 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592107417323175936
[junit4] 2> 1938387 INFO (qtp591617657-19199) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard5: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1938390 INFO (searcherExecutor-7023-thread-1-processing-n:127.0.0.1:36880_solr x:replacenodetest_coll_notarget_shard5_replica_n16 s:shard5 c:replacenodetest_coll_notarget r:core_node19) [n:127.0.0.1:36880_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore [replacenodetest_coll_notarget_shard5_replica_n16] Registered new searcher Searcher@7ad5a633[replacenodetest_coll_notarget_shard5_replica_n16] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1938400 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1938400 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1938408 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1938408 INFO (qtp798278088-19177) [n:127.0.0.1:48451_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1938409 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1938409 INFO (qtp1085953888-19216) [n:127.0.0.1:35902_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=
[...truncated too long message...]
collections/collection1/state.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 5 /solr/overseer_elect/election/73276831123111941-127.0.0.1:40099_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> 2202882 INFO (SUITE-TestSkipOverseerOperations-seed#[E0A318B8151D8406]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 11, 2018 12:30:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> Feb 11, 2018 12:30:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSkipOverseerOperations:
[junit4] 2> 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 11, 2018 12:30:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
[junit4] 2> Feb 11, 2018 12:30:40 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=513, maxMBSortInHeap=6.289298543473171, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-QA, timezone=Pacific/Honolulu
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=256489992,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [BadIndexSchemaTest, TestXmlQParser, UpdateParamsTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestRestoreCore, ConcurrentCreateRoutedAliasTest, DistributedVersionInfoTest, TestMultiValuedNumericRangeQuery, TestConfig, TestLuceneIndexBackCompat, AddBlockUpdateTest, ZkStateReaderTest, TestDynamicFieldResource, TestFreeTextSuggestions, TestCollectionAPI, TestWriterPerf, HttpTriggerListenerTest, SolrIndexConfigTest, CdcrRequestHandlerTest, NodeLostTriggerTest, TestIndexSearcher, TestFastOutputStream, OverseerTaskQueueTest, TestCryptoKeys, TestSchemalessBufferedUpdates, TestSolrQueryParser, TestInPlaceUpdatesDistrib, TestGroupingSearch, MissingSegmentRecoveryTest, TestUnifiedSolrHighlighter, TestBinaryField, TestStressInPlaceUpdates, CreateCollectionCleanupTest, OpenCloseCoreStressTest, FieldAnalysisRequestHandlerTest, TestCloudManagedSchema, LeaderInitiatedRecoveryOnShardRestartTest, TestCSVLoader, TestInitParams, RAMDirectoryFactoryTest, HighlighterConfigTest, MergeStrategyTest, TestObjectReleaseTracker, ZkCLITest, SolrCoreCheckLockOnStartupTest, TestSearcherReuse, DistributedSuggestComponentTest, TestIndexingPerformance, DistributedFacetPivotLongTailTest, TestFieldTypeCollectionResource, TestDocSet, BasicAuthIntegrationTest, TestLegacyFieldReuse, TestManagedResource, TestDocumentBuilder, TestQueryWrapperFilter, FullSolrCloudDistribCmdsTest, CollectionsAPIDistributedZkTest, PluginInfoTest, DistanceUnitsTest, TestReRankQParserPlugin, DataDrivenBlockJoinTest, TestPullReplica, SolrRequestParserTest, TestSolrConfigHandlerCloud, TestPushWriter, HdfsChaosMonkeySafeLeaderTest, TestFilteredDocIdSet, ActionThrottleTest, CdcrBootstrapTest, GraphQueryTest, AutoAddReplicasIntegrationTest, TestDownShardTolerantSearch, SolrCLIZkUtilsTest, DebugComponentTest, TestGenericDistributedQueue, ZkNodePropsTest, TestIBSimilarityFactory, DateRangeFieldTest, SpellCheckCollatorTest, BlockJoinFacetDistribTest, TestStressLucene, TestTestInjection, UUIDFieldTest, TestSubQueryTransformer, TestComponentsName, TestLocalFSCloudBackupRestore, TestWordDelimiterFilterFactory, TestExactSharedStatsCache, TestFieldCacheReopen, DistribDocExpirationUpdateProcessorTest, TestSSLRandomization, TestCSVResponseWriter, ManagedSchemaRoundRobinCloudTest, TestLMDirichletSimilarityFactory, MetricsConfigTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestOrdValues, SuggesterWFSTTest, RuleEngineTest, TestNodeLostTrigger, TlogReplayBufferedWhileIndexingTest, TestSolrCloudWithKerberosAlt, TestAddFieldRealTimeGet, TestNumericTokenStream, ShardRoutingTest, TestAuthenticationFramework, TestSolr4Spatial2, TestMacros, ExternalFileFieldSortTest, HttpPartitionOnCommitTest, TestEmbeddedSolrServerSchemaAPI, V2ApiIntegrationTest, SimpleFacetsTest, SSLMigrationTest, SolrGraphiteReporterTest, TestSerializedLuceneMatchVersion, BigEndianAscendingWordDeserializerTest, TestOverriddenPrefixQueryForCustomFieldType, TestManagedSchemaThreadSafety, HdfsLockFactoryTest, TestPhraseSuggestions, TestComputePlanAction, SolrIndexSplitterTest, TestRemoteStreaming, TestAnalyzeInfixSuggestions, TestHighlightDedupGrouping, BasicDistributedZkTest, TestRandomRequestDistribution, TestSegmentSorting, TestSkipOverseerOperations]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSkipOverseerOperations -Dtests.seed=E0A318B8151D8406 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=Pacific/Honolulu -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestSkipOverseerOperations (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSkipOverseerOperations:
[junit4] > 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([E0A318B8151D8406]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=17954, name=qtp1019377703-17954, state=TIMED_WAITING, group=TGRP-TestSkipOverseerOperations]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([E0A318B8151D8406]:0)
[junit4] Completed [570/775 (2!)] on J0 in 69.63s, 2 tests, 2 errors <<< FAILURES!
[...truncated 49022 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 368 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/368/
8 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=5571, name=qtp1144427281-5571, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=5571, name=qtp1144427281-5571, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B1CB060F4BDB9291]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=5571, name=qtp1144427281-5571, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=5571, name=qtp1144427281-5571, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B1CB060F4BDB9291]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.security.PKIAuthenticationIntegrationTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.security.PKIAuthenticationIntegrationTest: 1) Thread[id=4655, name=qtp566231205-4655, state=TIMED_WAITING, group=TGRP-PKIAuthenticationIntegrationTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.security.PKIAuthenticationIntegrationTest:
1) Thread[id=4655, name=qtp566231205-4655, state=TIMED_WAITING, group=TGRP-PKIAuthenticationIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B1CB060F4BDB9291]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.security.PKIAuthenticationIntegrationTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=4655, name=qtp566231205-4655, state=TIMED_WAITING, group=TGRP-PKIAuthenticationIntegrationTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=4655, name=qtp566231205-4655, state=TIMED_WAITING, group=TGRP-PKIAuthenticationIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B1CB060F4BDB9291]:0)
FAILED: org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([9277CC690D14BCB5:B0B74D922E7E96A5]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream(StreamExpressionTest.java:9435)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.StreamExpressionTest
Error Message:
11 threads leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.StreamExpressionTest: 1) Thread[id=2309, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2305, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2530, name=zkConnectionManagerCallback-992-thread-1-SendThread(127.0.0.1:42266), state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 4) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2320, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=2531, name=zkConnectionManagerCallback-992-thread-1-EventThread, state=WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 9) Thread[id=2319, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=2302, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=2308, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 11 threads leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.StreamExpressionTest:
1) Thread[id=2309, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2305, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2530, name=zkConnectionManagerCallback-992-thread-1-SendThread(127.0.0.1:42266), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
4) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=2320, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=2531, name=zkConnectionManagerCallback-992-thread-1-EventThread, state=WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
9) Thread[id=2319, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=2302, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=2308, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([9277CC690D14BCB5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.StreamExpressionTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2305, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2530, name=zkConnectionManagerCallback-992-thread-1-SendThread(127.0.0.1:42266), state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 3) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2305, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2530, name=zkConnectionManagerCallback-992-thread-1-SendThread(127.0.0.1:42266), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
3) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([9277CC690D14BCB5]:0)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([B1CB060F4BDB9291:DC37A2F2F1936D96]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12229 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> 210228 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_B1CB060F4BDB9291-001/init-core-data-001
[junit4] 2> 210246 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
[junit4] 2> 210246 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 210247 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 210249 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_B1CB060F4BDB9291-001/tempDir-001
[junit4] 2> 210249 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 210249 INFO (Thread-140) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 210249 INFO (Thread-140) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 210294 ERROR (Thread-140) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 210371 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:35708
[junit4] 2> 210489 INFO (zkConnectionManagerCallback-188-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210511 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 210530 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 210540 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 210557 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 210557 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 210557 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 210557 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13a83b4a{/solr,null,AVAILABLE}
[junit4] 2> 210559 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@338e7211{HTTP/1.1,[http/1.1]}{127.0.0.1:34132}
[junit4] 2> 210559 INFO (jetty-launcher-185-thread-2) [ ] o.e.j.s.Server Started @220954ms
[junit4] 2> 210559 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34132}
[junit4] 2> 210560 ERROR (jetty-launcher-185-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 210560 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 210560 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 210560 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 210560 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T06:11:40.258Z
[junit4] 2> 210563 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 210563 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 210563 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 210564 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33fbdb66{/solr,null,AVAILABLE}
[junit4] 2> 210564 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3c75c8d{HTTP/1.1,[http/1.1]}{127.0.0.1:43266}
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.e.j.s.Server Started @220960ms
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43266}
[junit4] 2> 210565 ERROR (jetty-launcher-185-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 210565 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T06:11:40.263Z
[junit4] 2> 210611 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 210611 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 210611 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 210611 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@199244c4{/solr,null,AVAILABLE}
[junit4] 2> 210612 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1f43555c{HTTP/1.1,[http/1.1]}{127.0.0.1:42992}
[junit4] 2> 210612 INFO (jetty-launcher-185-thread-3) [ ] o.e.j.s.Server Started @221006ms
[junit4] 2> 210612 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42992}
[junit4] 2> 210612 ERROR (jetty-launcher-185-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 210613 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 210613 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 210613 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 210613 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T06:11:40.311Z
[junit4] 2> 210623 INFO (zkConnectionManagerCallback-194-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210636 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 210651 INFO (zkConnectionManagerCallback-192-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210652 INFO (zkConnectionManagerCallback-190-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210655 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 210655 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 210704 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10027c019ad0001, likely client has closed socket
[junit4] 2> 210766 INFO (jetty-launcher-185-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35708/solr
[junit4] 2> 210778 INFO (jetty-launcher-185-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35708/solr
[junit4] 2> 210810 INFO (jetty-launcher-185-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35708/solr
[junit4] 2> 210847 INFO (zkConnectionManagerCallback-202-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210859 INFO (zkConnectionManagerCallback-198-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210942 INFO (zkConnectionManagerCallback-206-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 210956 INFO (zkConnectionManagerCallback-208-thread-1-processing-n:127.0.0.1:34132_solr) [n:127.0.0.1:34132_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 211009 INFO (zkConnectionManagerCallback-210-thread-1-processing-n:127.0.0.1:42992_solr) [n:127.0.0.1:42992_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 211012 INFO (zkConnectionManagerCallback-212-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 211931 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 211932 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43266_solr
[junit4] 2> 211950 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer Overseer (id=72101300055900169-127.0.0.1:43266_solr-n_0000000000) starting
[junit4] 2> 211967 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 211968 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34132_solr
[junit4] 2> 212067 INFO (zkCallback-207-thread-1-processing-n:127.0.0.1:34132_solr) [n:127.0.0.1:34132_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 212083 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 212173 INFO (zkCallback-211-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 212199 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 212201 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42992_solr
[junit4] 2> 212266 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 212303 INFO (zkCallback-207-thread-1-processing-n:127.0.0.1:34132_solr) [n:127.0.0.1:34132_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 212303 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 212412 INFO (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 212412 INFO (zkCallback-209-thread-1-processing-n:127.0.0.1:42992_solr) [n:127.0.0.1:42992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 212428 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 212446 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:42992_solr, 127.0.0.1:34132_solr]
[junit4] 2> 212446 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 212446 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 212447 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43266_solr
[junit4] 2> 212499 INFO (zkCallback-207-thread-1-processing-n:127.0.0.1:34132_solr) [n:127.0.0.1:34132_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 212500 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 212500 DEBUG (OverseerAutoScalingTriggerThread-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 212500 DEBUG (ScheduledTrigger-293-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 212505 INFO (zkCallback-209-thread-1-processing-n:127.0.0.1:42992_solr) [n:127.0.0.1:42992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 212506 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:42992_solr"} current state version: 0
[junit4] 2> 212506 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:42992_solr
[junit4] 2> 212506 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:43266_solr"} current state version: 0
[junit4] 2> 212506 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:43266_solr
[junit4] 2> 212506 INFO (zkCallback-211-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 212829 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34132.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213028 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34132.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213061 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34132.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213063 INFO (jetty-launcher-185-thread-2) [n:127.0.0.1:34132_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_B1CB060F4BDB9291-001/tempDir-001/node2/.
[junit4] 2> 213120 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42992.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213229 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42992.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213229 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42992.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213230 INFO (jetty-launcher-185-thread-3) [n:127.0.0.1:42992_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_B1CB060F4BDB9291-001/tempDir-001/node3/.
[junit4] 2> 213299 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43266.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213351 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43266.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213351 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43266.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@366735cc
[junit4] 2> 213352 INFO (jetty-launcher-185-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_B1CB060F4BDB9291-001/tempDir-001/node1/.
[junit4] 2> 213501 DEBUG (ScheduledTrigger-293-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 214071 INFO (zkConnectionManagerCallback-220-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 214440 INFO (zkConnectionManagerCallback-224-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 214616 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 214616 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[B1CB060F4BDB9291]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35708/solr ready
[junit4] 2> 214680 DEBUG (ScheduledTrigger-293-thread-2) [n:127.0.0.1:43266_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 214746 INFO (qtp622348174-919) [n:127.0.0.1:43266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 215264 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 215266 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 215268 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 215282 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 215283 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 215284 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 215302 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 215303 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 215338 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 215339 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 215341 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 215378 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 215380 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 215381 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 215432 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 215433 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 215452 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 215453 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 215467 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 215469 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 215470 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 215505 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 215507 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 215524 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 215525 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 215527 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 215540 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 215541 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 215543 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 215544 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 215562 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 215563 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 215564 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 215579 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 215597 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 215599 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 215600 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 215614 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 215615 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 215617 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 215635 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 215636 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 215650 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 215651 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 215652 INFO (OverseerThreadFactory-296-thread-1-processing-n:127.0.0.1:43266_solr) [n:127.0.0.1:43266_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 215686 DEBUG (ScheduledTrigger-293-thread-1) [n:127.0.0.1:43266_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 215745 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 215745 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 215746 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 215877 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 215877 INFO (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 215878 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 215879 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 215879 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 215879 INFO (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 215880 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 215880 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 215880 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 215880 INFO (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 215881 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 215893 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 216003 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 216158 INFO (qtp200377097-940) [n:127.0.0.1:42992_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 216159 INFO (qtp200377097-940) [n:127.0.0.1:42992_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 216297 INFO (qtp1241840354-918) [n:127.0.0.1:34132_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 216297 INFO (qtp1241840354-918) [n:127.0.0.1:34132_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 216543 INFO (qtp622348174-933) [n:127.0.0.1:43266_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n3&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 216544 INFO (qtp622348174-933) [n:127.0.0.1:43266_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 216597 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "node_name":"127.0.0.1:42992_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 216597 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "node_name":"127.0.0.1:42992_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 216597 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"http://127.0.0.1:42992/solr","node_name":"127.0.0.1:42992_solr","state":"down","type":"NRT"}
[junit4] 2> 216597 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "node_name":"127.0.0.1:42992_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 216622 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "node_name":"127.0.0.1:34132_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 216622 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "node_name":"127.0.0.1:34132_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 216622 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"http://127.0.0.1:34132/solr","node_name":"127.0.0.1:34132_solr","state":"down","type":"NRT"}
[junit4] 2> 216622 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42992/solr",
[junit4] 2> "node_name":"127.0.0.1:42992_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:34132/solr",
[junit4] 2> "node_name":"127.0.0.1:34132_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 216622 DEBUG (OverseerStateUpdate-72101300055900169-127.0.0.1:43266_solr-n_0000000000) [n:127.0.0.1:43266_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43266/solr",
[...truncated too long message...]
che.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 4) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=2320, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 7) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 8) Thread[id=2531, name=zkConnectionManagerCallback-992-thread-1-EventThread, state=WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4] > 9) Thread[id=2319, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 10) Thread[id=2302, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 11) Thread[id=2308, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([9277CC690D14BCB5]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=2305, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=2530, name=zkConnectionManagerCallback-992-thread-1-SendThread(127.0.0.1:42266), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
[junit4] > 3) Thread[id=3609, name=zkCallback-991-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=3763, name=zkCallback-991-thread-6, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=3764, name=zkCallback-991-thread-7, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([9277CC690D14BCB5]:0)
[junit4] Completed [136/146 (1!)] on J0 in 284.09s, 153 tests, 1 failure, 2 errors, 9 skipped <<< FAILURES!
[...truncated 47446 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 367 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/367/
7 tests failed.
FAILED: org.apache.solr.cloud.CreateRoutedAliasTest.testTimezoneAbsoluteDate
Error Message:
Error from server at http://127.0.0.1:37212/solr: Collection : testV2_2018-02-10 is part of alias testV2 remove or modify the alias before removing this collection.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37212/solr: Collection : testV2_2018-02-10 is part of alias testV2 remove or modify the alias before removing this collection.
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:ACD3F07892DFF673]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:444)
at org.apache.solr.cloud.CreateRoutedAliasTest.doBefore(CreateRoutedAliasTest.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:8167B2CFA864C2F5]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 8 in http://127.0.0.1:46771/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 8 in http://127.0.0.1:46771/solr
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:C8C3F4B92BC865AA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:885)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLost
Error Message:
org.apache.solr.common.SolrException:
Stack Trace:
org.apache.solr.common.SolrException: org.apache.solr.common.SolrException:
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:B62643EB8572CA8B]:0)
at org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:78)
at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getNodeValues(SolrClientNodeStateProvider.java:111)
at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.printState(ComputePlanActionTest.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException:
at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$AutoScalingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:250)
at org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:76)
... 39 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:38738/solr
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:650)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:292)
at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$AutoScalingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:227)
... 40 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:38738 [/127.0.0.1] failed: Connection refused (Connection refused)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 45 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
... 55 more
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:39311_solr, 127.0.0.1:44537_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node3/data/", "base_url":"https://127.0.0.1:39311/solr", "node_name":"127.0.0.1:39311_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node3/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node5":{ "dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node5/data/", "base_url":"https://127.0.0.1:39311/solr", "node_name":"127.0.0.1:39311_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node5/data/tlog", "core":"testSimple1_shard1_replica_n2", "shared_storage":"true", "state":"active"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node7/data/", "base_url":"https://127.0.0.1:39311/solr", "node_name":"127.0.0.1:39311_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node7/data/tlog", "core":"testSimple1_shard2_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node8/data/", "base_url":"https://127.0.0.1:38878/solr", "node_name":"127.0.0.1:38878_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n6", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:39311_solr, 127.0.0.1:44537_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node3/data/",
"base_url":"https://127.0.0.1:39311/solr",
"node_name":"127.0.0.1:39311_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node3/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node5":{
"dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node5/data/",
"base_url":"https://127.0.0.1:39311/solr",
"node_name":"127.0.0.1:39311_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node5/data/tlog",
"core":"testSimple1_shard1_replica_n2",
"shared_storage":"true",
"state":"active"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node7/data/",
"base_url":"https://127.0.0.1:39311/solr",
"node_name":"127.0.0.1:39311_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node7/data/tlog",
"core":"testSimple1_shard2_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node8/data/",
"base_url":"https://127.0.0.1:38878/solr",
"node_name":"127.0.0.1:38878_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:33146/data/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n6",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:3180A9EB216B7BDC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:388DE0F17832DAFF]:0)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown(TestTriggerIntegration.java:1102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([9338D150698AF0D:64CF29E8BCD0500A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12134 lines...]
[junit4] Suite: org.apache.solr.cloud.ReplaceNodeNoTargetTest
[junit4] 2> 79901 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/init-core-data-001
[junit4] 2> 79902 WARN (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
[junit4] 2> 79902 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 79938 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 79954 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001
[junit4] 2> 79977 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 80011 INFO (Thread-13) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 80011 INFO (Thread-13) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 80294 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:36210
[junit4] 2> 81026 ERROR (Thread-13) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 81264 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 83004 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83085 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83085 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83085 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83040 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83056 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 83358 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83358 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83394 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 83447 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83492 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83492 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 83593 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83593 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83593 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 83695 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83695 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83695 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 83804 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83804 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83804 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 83865 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 83865 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 83865 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 83974 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@dba13ad{/solr,null,AVAILABLE}
[junit4] 2> 84032 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f1fe86{/solr,null,AVAILABLE}
[junit4] 2> 84048 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3db19308{/solr,null,AVAILABLE}
[junit4] 2> 84049 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b2a3cb1{/solr,null,AVAILABLE}
[junit4] 2> 84049 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fcba3fd{/solr,null,AVAILABLE}
[junit4] 2> 83975 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6889a484{/solr,null,AVAILABLE}
[junit4] 2> 84145 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@216be34b{HTTP/1.1,[http/1.1]}{127.0.0.1:45005}
[junit4] 2> 84145 INFO (jetty-launcher-25-thread-5) [ ] o.e.j.s.Server Started @94377ms
[junit4] 2> 84182 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e28d2f9{HTTP/1.1,[http/1.1]}{127.0.0.1:43694}
[junit4] 2> 84182 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45005}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@18c936c3{HTTP/1.1,[http/1.1]}{127.0.0.1:41887}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-4) [ ] o.e.j.s.Server Started @94427ms
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a381ef8{HTTP/1.1,[http/1.1]}{127.0.0.1:40677}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-3) [ ] o.e.j.s.Server Started @94428ms
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4952db4e{HTTP/1.1,[http/1.1]}{127.0.0.1:41353}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-6) [ ] o.e.j.s.Server Started @94428ms
[junit4] 2> 84158 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61803c04{HTTP/1.1,[http/1.1]}{127.0.0.1:37129}
[junit4] 2> 84199 INFO (jetty-launcher-25-thread-2) [ ] o.e.j.s.Server Started @94430ms
[junit4] 2> 84182 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.Server Started @94414ms
[junit4] 2> 84255 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43694}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40677}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41353}
[junit4] 2> 84199 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37129}
[junit4] 2> 84196 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41887}
[junit4] 2> 84447 ERROR (jetty-launcher-25-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84448 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84448 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84448 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84448 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.057Z
[junit4] 2> 84467 ERROR (jetty-launcher-25-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84468 ERROR (jetty-launcher-25-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84468 ERROR (jetty-launcher-25-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84468 ERROR (jetty-launcher-25-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84475 ERROR (jetty-launcher-25-thread-6) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84540 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84540 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84540 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84541 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84541 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 84557 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84557 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84557 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.166Z
[junit4] 2> 84557 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.167Z
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.167Z
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.167Z
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84558 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T22:48:21.167Z
[junit4] 2> 84573 INFO (zkConnectionManagerCallback-30-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 84703 INFO (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 84740 INFO (zkConnectionManagerCallback-38-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 84757 INFO (zkConnectionManagerCallback-34-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 84831 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 84841 INFO (zkConnectionManagerCallback-40-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 84904 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 85012 INFO (zkConnectionManagerCallback-36-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 85230 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 85230 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 85244 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 85246 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100262a2ba40002, likely client has closed socket
[junit4] 2> 85309 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 85452 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100262a2ba40006, likely client has closed socket
[junit4] 2> 85696 INFO (jetty-launcher-25-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 85741 INFO (jetty-launcher-25-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 85814 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 85768 INFO (jetty-launcher-25-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 86160 INFO (jetty-launcher-25-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 86161 INFO (jetty-launcher-25-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36210/solr
[junit4] 2> 86604 INFO (zkConnectionManagerCallback-62-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 86662 INFO (zkConnectionManagerCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 86622 INFO (zkConnectionManagerCallback-54-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 86605 INFO (zkConnectionManagerCallback-64-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 86605 INFO (zkConnectionManagerCallback-60-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 86604 INFO (zkConnectionManagerCallback-58-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87229 INFO (zkConnectionManagerCallback-72-thread-1-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87245 INFO (zkConnectionManagerCallback-74-thread-1-processing-n:127.0.0.1:41887_solr) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87245 INFO (zkConnectionManagerCallback-76-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87250 INFO (zkConnectionManagerCallback-66-thread-1-processing-n:127.0.0.1:41353_solr) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87251 INFO (zkConnectionManagerCallback-70-thread-1-processing-n:127.0.0.1:37129_solr) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87253 INFO (zkConnectionManagerCallback-68-thread-1-processing-n:127.0.0.1:40677_solr) [n:127.0.0.1:40677_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 89463 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 89478 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43694_solr
[junit4] 2> 89589 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.c.Overseer Overseer (id=72099556600578065-127.0.0.1:43694_solr-n_0000000000) starting
[junit4] 2> 91750 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43694_solr
[junit4] 2> 92180 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 94088 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 94163 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 94290 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45005_solr
[junit4] 2> 94363 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 94473 INFO (zkCallback-71-thread-1-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 95448 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43694.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 95501 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43694.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 95502 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43694.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 95539 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:43694_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node1/.
[junit4] 2> 95882 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 95899 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 95994 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 95994 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 96010 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 96047 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41353_solr
[junit4] 2> 96065 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37129_solr
[junit4] 2> 96066 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 96191 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 96244 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 96247 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 96337 INFO (zkCallback-65-thread-1-processing-n:127.0.0.1:41353_solr) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 96398 INFO (zkCallback-71-thread-1-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 96536 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 96544 INFO (zkCallback-69-thread-1-processing-n:127.0.0.1:37129_solr) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 96581 INFO (zkCallback-73-thread-1-processing-n:127.0.0.1:41887_solr) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 96800 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40677_solr
[junit4] 2> 96865 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41887_solr
[junit4] 2> 96911 INFO (zkCallback-65-thread-1-processing-n:127.0.0.1:41353_solr) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 96914 INFO (zkCallback-75-thread-2-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 96927 INFO (zkCallback-73-thread-1-processing-n:127.0.0.1:41887_solr) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 96930 INFO (zkCallback-69-thread-1-processing-n:127.0.0.1:37129_solr) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 96942 INFO (zkCallback-71-thread-1-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 96946 INFO (zkCallback-65-thread-1-processing-n:127.0.0.1:41353_solr) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 96947 INFO (zkCallback-73-thread-1-processing-n:127.0.0.1:41887_solr) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 96951 INFO (zkCallback-75-thread-2-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 97099 INFO (zkCallback-67-thread-1-processing-n:127.0.0.1:40677_solr) [n:127.0.0.1:40677_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 97170 INFO (zkCallback-69-thread-1-processing-n:127.0.0.1:37129_solr) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 98440 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45005.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 98460 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37129.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 98871 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45005.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 98872 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45005.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 99022 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37129.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 99022 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37129.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 99041 INFO (jetty-launcher-25-thread-5) [n:127.0.0.1:45005_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node5/.
[junit4] 2> 99130 INFO (jetty-launcher-25-thread-2) [n:127.0.0.1:37129_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node2/.
[junit4] 2> 99717 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41887.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 100126 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41887.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 100127 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41887.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 100128 INFO (jetty-launcher-25-thread-4) [n:127.0.0.1:41887_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node4/.
[junit4] 2> 100663 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40677.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101085 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41353.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101149 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40677.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101149 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40677.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101151 INFO (jetty-launcher-25-thread-3) [n:127.0.0.1:40677_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node3/.
[junit4] 2> 101276 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41353.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101276 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41353.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@364740c4
[junit4] 2> 101311 INFO (jetty-launcher-25-thread-6) [n:127.0.0.1:41353_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_9338D150698AF0D-001/tempDir-001/node6/.
[junit4] 2> 104372 INFO (zkConnectionManagerCallback-90-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 104908 INFO (zkConnectionManagerCallback-94-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 105092 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 105094 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[9338D150698AF0D]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36210/solr ready
[junit4] 2> 105541 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[9338D150698AF0D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 105680 INFO (zkConnectionManagerCallback-96-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 105880 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[9338D150698AF0D]) [ ] o.a.s.c.ReplaceNodeNoTargetTest total_jettys: 6
[junit4] 2> 108716 INFO (qtp1196998372-166) [n:127.0.0.1:37129_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=319
[junit4] 2> 109299 INFO (qtp995330576-175) [n:127.0.0.1:40677_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 109774 INFO (qtp2103406526-140) [n:127.0.0.1:41353_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=107
[junit4] 2> 109978 INFO (qtp2088020688-157) [n:127.0.0.1:41887_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 110047 INFO (qtp18668090-148) [n:127.0.0.1:43694_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=16
[junit4] 2> 110068 INFO (qtp724530558-136) [n:127.0.0.1:45005_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 110144 INFO (qtp724530558-132) [n:127.0.0.1:45005_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=2493
[junit4] 2> 110198 INFO (qtp1196998372-170) [n:127.0.0.1:37129_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&collection.configName=conf1&name=replacenodetest_coll_notarget&nrtReplicas=2&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 110399 INFO (OverseerThreadFactory-175-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll_notarget
[junit4] 2> 110602 INFO (qtp1196998372-171) [n:127.0.0.1:37129_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 110710 INFO (qtp995330576-179) [n:127.0.0.1:40677_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 110731 INFO (qtp2103406526-144) [n:127.0.0.1:41353_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 110766 INFO (qtp2088020688-157) [n:127.0.0.1:41887_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=17
[junit4] 2> 110800 INFO (qtp18668090-152) [n:127.0.0.1:43694_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 110856 INFO (qtp724530558-138) [n:127.0.0.1:45005_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 111153 INFO (OverseerThreadFactory-175-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 640328699 sessionWrapper.createTime 640328420660298, this.sessionWrapper.createTime 640328420660298
[junit4] 2> 111486 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45005/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111523 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43694/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111525 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41887/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111649 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41353/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111687 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40677/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111772 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37129/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111917 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n13",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37129/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111966 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n15",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40677/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 111989 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n17",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41353/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 112006 INFO (OverseerStateUpdate-72099556600578065-127.0.0.1:43694_solr-n_0000000000) [n:127.0.0.1:43694_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41887/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 112199 INFO (qtp724530558-134) [n:127.0.0.1:45005_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112199 INFO (qtp724530558-134) [n:127.0.0.1:45005_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112250 INFO (qtp2103406526-146) [n:127.0.0.1:41353_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n17&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112364 INFO (qtp2103406526-146) [n:127.0.0.1:41353_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112349 INFO (qtp995330576-179) [n:127.0.0.1:40677_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node16&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n15&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112365 INFO (qtp995330576-179) [n:127.0.0.1:40677_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112212 INFO (qtp18668090-152) [n:127.0.0.1:43694_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112365 INFO (qtp18668090-152) [n:127.0.0.1:43694_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112211 INFO (qtp1196998372-171) [n:127.0.0.1:37129_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n10&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112366 INFO (qtp1196998372-171) [n:127.0.0.1:37129_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112348 INFO (qtp1196998372-168) [n:127.0.0.1:37129_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node14&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n13&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112347 INFO (qtp2088020688-162) [n:127.0.0.1:41887_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node20&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n18&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112367 INFO (qtp2088020688-162) [n:127.0.0.1:41887_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 112347 INFO (qtp995330576-181) [n:127.0.0.1:40677_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n8&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112346 INFO (qtp2103406526-144) [n:127.0.0.1:41353_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n5&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112210 INFO (qtp2088020688-157) [n:127.0.0.1:41887_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n3&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 112780 INFO (zkCallback-73-thread-1-processing-n:127.0.0.1:41887_solr) [n:127.0.0.1:41887_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 112781 INFO (zkCallback-69-thread-1-processing-n:127.0.0.1:37129_solr) [n:127.0.0.1:37129_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 112781 INFO (zkCallback-67-thread-1-processing-n:127.0.0.1:40677_solr) [n:127.0.0.1:40677_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 112781 INFO (zkCallback-71-thread-1-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 112782 INFO (zkCallback-75-thread-2-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 112783 INFO (zkCallback-65-thread-1-processing-n:127.0.0.1:41353_solr) [n:127.0.0.1:41353_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 113109 INFO (zkCallback-71-thread-2-processing-n:127.0.0.1:45005_solr) [n:127.0.0.1:45005_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 113115 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:43694_solr) [n:127.0.0.1:43694_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 113498 INFO (qtp2088020688-162) [n:127.0.0.1:41887_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 113598 INFO (qtp1196998372-171) [n:127.0.0.1:37129_solr c:replacenodetest_coll_notarget s:shard3 r:core_node12 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 113843 INFO (qtp2103406526-144) [n:127.0.0.1:41353_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 113852 INFO (qtp1196998372-171) [n:127.0.0.1:37129_solr c:replacenodetest_coll_notarget s:shard3 r:core_node12 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n10] Schema name=cloud-dynamic
[junit4] 2> 113945 INFO (qtp2088020688-162) [n:127.0.0.1:41887_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n18] Schema name=cloud-dynamic
[junit4] 2> 114028 INFO (qtp2103406526-144) [n:127.0.0.1:41353_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n5] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n5] Schema name=cloud-dynamic
[junit4] 2> 114054 INFO (qtp2103406526-146) [n:127
[...truncated too long message...]
olr/collections/tlog_replica_test_create_delete/terms/shard2
[junit4] 2> 24 /solr/collections/tlog_replica_test_create_delete/terms/shard1
[junit4] 2> 12 /solr/collections/tlog_replica_test_remove_leader/terms/shard1
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 11 /solr/collections/tlog_replica_test_recovery/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_real_time_get/terms/shard1
[junit4] 2> 9 /solr/collections/tlog_replica_test_kill_leader/terms/shard1
[junit4] 2> 9 /solr/security.json
[junit4] 2> 8 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
[junit4] 2> 8 /solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
[junit4] 2> 8 /solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
[junit4] 2> 7 /solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
[junit4] 2> 7 /solr/collections/tlog_replica_test_delete_by_id/terms/shard1
[junit4] 2> 5 /solr/collections/tlog_replica_test_add_docs/terms/shard1
[junit4] 2> 5 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 46 /solr/collections/tlog_replica_test_create_delete/state.json
[junit4] 2> 37 /solr/collections/tlog_replica_test_recovery/state.json
[junit4] 2> 31 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
[junit4] 2> 26 /solr/collections/tlog_replica_test_remove_leader/state.json
[junit4] 2> 26 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
[junit4] 2> 25 /solr/collections/tlog_replica_test_kill_leader/state.json
[junit4] 2> 24 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
[junit4] 2> 24 /solr/collections/tlog_replica_test_basic_leader_election/state.json
[junit4] 2> 15 /solr/collections/tlog_replica_test_add_docs/state.json
[junit4] 2> 15 /solr/collections/tlog_replica_test_real_time_get/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_delete_by_id/state.json
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 5 /solr/autoscaling.json
[junit4] 2> 4 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72099823368470539-core_node3-n_0000000000
[junit4] 2> 3 /solr/overseer_elect/election/72099823368470539-127.0.0.1:34384_solr-n_0000000002
[junit4] 2> 2 /solr/collections/tlog_replica_test_real_time_get/leader_elect/shard1/election/72099823368470533-core_node6-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72099823368470539-core_node11-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/72099823368470552-127.0.0.1:34384_solr-n_0000000006
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72099823368470539-core_node11-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/live_nodes
[junit4] 2> 11 /solr/collections
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 5 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 5 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_9338D150698AF0D-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {foo=PostingsFormat(name=MockRandom), title_s=PostingsFormat(name=MockRandom), id=PostingsFormat(name=LuceneVarGapFixedInterval), foo_s=PostingsFormat(name=MockRandom)}, docValues:{_version_=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70), inplace_updatable_int=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1370, maxMBSortInHeap=5.414837716141696, sim=RandomSimilarity(queryNorm=false): {}, locale=fr-BE, timezone=Australia/Eucla
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=189953040,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [TestBinaryField, TestSubQueryTransformer, CdcrRequestHandlerTest, TestHighlightDedupGrouping, TestUseDocValuesAsStored2, TestDocumentBuilder, TestWordDelimiterFilterFactory, SpellCheckCollatorTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestDynamicLoading, TestIBSimilarityFactory, TestSolrQueryParser, TestCloudRecovery, TestDocSet, DistanceUnitsTest, LeaderInitiatedRecoveryOnShardRestartTest, TestLocalFSCloudBackupRestore, TestBlobHandler, ZkFailoverTest, TestPhraseSuggestions, HdfsBasicDistributedZk2Test, AssignBackwardCompatibilityTest, ShardRoutingTest, TestIndexingPerformance, IndexBasedSpellCheckerTest, TlogReplayBufferedWhileIndexingTest, AddBlockUpdateTest, DistributedExpandComponentTest, TestGeoJSONResponseWriter, DirectUpdateHandlerTest, TestGroupingSearch, TestStressInPlaceUpdates, TestNumericTerms32, TestTrackingShardHandlerFactory, NodeLostTriggerTest, RemoteQueryErrorTest, DistributedVersionInfoTest, AutoAddReplicasIntegrationTest, BinaryUpdateRequestHandlerTest, UpdateParamsTest, HdfsThreadLeakTest, TestCSVResponseWriter, HLLUtilTest, TestSchemaResource, PluginInfoTest, CreateCollectionCleanupTest, TestConfig, TestConfigSets, TestRetrieveFieldsOptimizer, TestMultiValuedNumericRangeQuery, BlockDirectoryTest, TestExpandComponent, TestRemoteStreaming, TestFilteredDocIdSet, TestQuerySenderNoQuery, TestOmitPositions, TestFieldCacheReopen, CollectionsAPIDistributedZkTest, TestTriggerIntegration, TestLegacyFieldReuse, TestStressLucene, OutOfBoxZkACLAndCredentialsProvidersTest, TestLuceneMatchVersion, SolrSlf4jReporterTest, TestCrossCoreJoin, TestComplexPhraseQParserPlugin, SuggesterWFSTTest, TestConfigSetsAPI, MoveReplicaTest, EchoParamsTest, SolrGraphiteReporterTest, TestPHPSerializedResponseWriter, CursorPagingTest, TestMacros, TestExtendedDismaxParser, ZkNodePropsTest, ComputePlanActionTest, TestFieldTypeCollectionResource, HdfsCollectionsAPIDistributedZkTest, TestIndexSearcher, CurrencyRangeFacetCloudTest, TestWriterPerf, TestLuceneIndexBackCompat, TestJsonFacetRefinement, TestAuthorizationFramework, HdfsRecoveryZkTest, TestPolicyCloud, CdcrBootstrapTest, CloudExitableDirectoryReaderTest, CloudMLTQParserTest, SpellingQueryConverterTest, TestDownShardTolerantSearch, ManagedSchemaRoundRobinCloudTest, HdfsDirectoryFactoryTest, TestRestoreCore, TestInPlaceUpdatesDistrib, GraphQueryTest, BasicAuthIntegrationTest, SolrIndexSplitterTest, DataDrivenBlockJoinTest, TestRandomFaceting, UUIDFieldTest, TestDistributedSearch, V2ApiIntegrationTest, TestPullReplica, TestDynamicFieldResource, DeleteShardTest, TestExactSharedStatsCache, FieldAnalysisRequestHandlerTest, StatsComponentTest, DistribDocExpirationUpdateProcessorTest, TestHdfsBackupRestoreCore, DistributedSuggestComponentTest, TestComponentsName, TestCollectionAPI, DistributedTermsComponentTest, TestApiFramework, TestLMDirichletSimilarityFactory, RequestLoggingTest, ZkCLITest, TestStandardQParsers, TestHashPartitioner, TestAuthenticationFramework, DeleteLastCustomShardedReplicaTest, MetricsConfigTest, DefaultValueUpdateProcessorTest, SyncSliceTest, OpenCloseCoreStressTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, ClusterStateUpdateTest, LeaderElectionTest, TestFaceting, TestRecovery, ZkControllerTest, TestStressReorder, TestReload, SimpleFacetsTest, SpellCheckComponentTest, TestSort, TestLazyCores, HighlighterTest, DistributedQueryElevationComponentTest, SimplePostToolTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, SpatialFilterTest, TestCSVLoader, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, DebugComponentTest, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, DocumentAnalysisRequestHandlerTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, LoggingHandlerTest, SolrPluginUtilsTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, JsonLoaderTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestStressRecovery, TestTolerantSearch, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ActionThrottleTest, AsyncCallRequestStatusResponseTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteReplicaTest, HttpPartitionOnCommitTest, LegacyCloudClusterPropTest, MigrateRouteKeyTest, MultiThreadedOCPTest, NodeMutatorTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerTaskQueueTest, ReplicationFactorTest, RollingRestartTest, SSLMigrationTest, SaslZkACLProviderTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestRandomRequestDistribution, TestSkipOverseerOperations, TestTlogReplica]
[junit4] Completed [573/775 (7!)] on J0 in 152.69s, 13 tests, 1 failure <<< FAILURES!
[...truncated 48874 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 366 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/366/
2 tests failed.
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:40822_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:40822_solr
at __randomizedtesting.SeedInfo.seed([688422546588007E:E0D01D8ECB746D86]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
Stack Trace:
java.lang.ClassCastException: org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
at __randomizedtesting.SeedInfo.seed([688422546588007E:A13160FA6CEFC68B]:0)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12140 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/init-core-data-001
[junit4] 2> 9712 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 9713 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 10032 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 10033 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001
[junit4] 2> 10033 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 10034 INFO (Thread-20) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 10034 INFO (Thread-20) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 10045 ERROR (Thread-20) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 10135 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:45763
[junit4] 2> 10142 INFO (zkConnectionManagerCallback-23-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10168 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 10173 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 10183 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 10183 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 10183 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 10187 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@172b8dc{/solr,null,AVAILABLE}
[junit4] 2> 10204 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 10204 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 10204 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 10205 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@645f3be9{/solr,null,AVAILABLE}
[junit4] 2> 10235 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@ac5c65b{SSL,[ssl, http/1.1]}{127.0.0.1:60585}
[junit4] 2> 10235 INFO (jetty-launcher-20-thread-2) [ ] o.e.j.s.Server Started @12301ms
[junit4] 2> 10235 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60585}
[junit4] 2> 10237 ERROR (jetty-launcher-20-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 10237 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 10238 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 10238 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 10238 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T19:11:25.106Z
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@191cd13e{SSL,[ssl, http/1.1]}{127.0.0.1:50598}
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.e.j.s.Server Started @12316ms
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50598}
[junit4] 2> 10250 ERROR (jetty-launcher-20-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 10250 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T19:11:25.118Z
[junit4] 2> 10260 INFO (zkConnectionManagerCallback-25-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10268 INFO (zkConnectionManagerCallback-27-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10268 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 10268 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 10333 INFO (jetty-launcher-20-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45763/solr
[junit4] 2> 10349 INFO (zkConnectionManagerCallback-31-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10351 INFO (jetty-launcher-20-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45763/solr
[junit4] 2> 10380 INFO (zkConnectionManagerCallback-35-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10381 INFO (zkConnectionManagerCallback-37-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10403 INFO (zkConnectionManagerCallback-39-thread-1-processing-n:127.0.0.1:60585_solr) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10513 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 10514 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50598_solr
[junit4] 2> 10515 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.Overseer Overseer (id=73272749152403461-127.0.0.1:50598_solr-n_0000000000) starting
[junit4] 2> 10523 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50598_solr
[junit4] 2> 10524 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 10526 INFO (OverseerStateUpdate-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 10536 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 10538 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50598_solr]
[junit4] 2> 10538 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 10538 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 10542 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 10544 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 10548 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60585_solr
[junit4] 2> 10548 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 10549 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 10557 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 10557 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 10573 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:60585_solr) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 10745 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50598.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10773 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60585.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10784 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50598.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10784 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50598.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10786 INFO (jetty-launcher-20-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node1/.
[junit4] 2> 10807 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60585.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10808 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60585.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 10839 INFO (jetty-launcher-20-thread-2) [n:127.0.0.1:60585_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node2/.
[junit4] 2> 10913 INFO (zkConnectionManagerCallback-45-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10930 INFO (zkConnectionManagerCallback-49-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10931 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 10934 INFO (SUITE-TriggerIntegrationTest-seed#[688422546588007E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45763/solr ready
[junit4] 2> 10956 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 10957 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 10957 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 10957 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 10973 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 10974 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 10974 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 11151 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=9
[junit4] 2> 11174 INFO (qtp91067448-101) [n:127.0.0.1:60585_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 11175 DEBUG (qtp1294856939-106) [n:127.0.0.1:50598_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 11176 INFO (qtp1294856939-106) [n:127.0.0.1:50598_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=69
[junit4] 2> 11177 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 11184 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:50598_solr]
[junit4] 2> 11184 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 11184 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 11187 INFO (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 11188 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 11188 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 11188 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 11189 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 11202 INFO (qtp1294856939-104) [n:127.0.0.1:50598_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 11209 INFO (qtp91067448-99) [n:127.0.0.1:60585_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 11210 DEBUG (qtp91067448-101) [n:127.0.0.1:60585_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 11210 INFO (qtp91067448-101) [n:127.0.0.1:60585_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 11211 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 11211 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:50598_solr]
[junit4] 2> 11211 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 11211 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:50598_solr]
[junit4] 2> 11211 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 11211 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 11213 INFO (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 11216 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 11216 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 11217 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 11217 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 11219 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 11221 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 11221 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 11221 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 11223 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e49624{/solr,null,AVAILABLE}
[junit4] 2> 11223 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1ed4e1ca{SSL,[ssl, http/1.1]}{127.0.0.1:36298}
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.Server Started @13290ms
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36298}
[junit4] 2> 11224 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 11224 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T19:11:26.092Z
[junit4] 2> 11311 INFO (zkConnectionManagerCallback-51-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11313 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 11337 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45763/solr
[junit4] 2> 11349 INFO (zkConnectionManagerCallback-55-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11358 INFO (zkConnectionManagerCallback-57-thread-1-processing-n:127.0.0.1:36298_solr) [n:127.0.0.1:36298_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11365 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 11367 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 11369 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36298_solr
[junit4] 2> 11370 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:60585_solr) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 11373 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 11376 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 11377 INFO (zkCallback-56-thread-1-processing-n:127.0.0.1:36298_solr) [n:127.0.0.1:36298_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 11533 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36298.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 11553 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36298.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 11553 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36298.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 11555 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [n:127.0.0.1:36298_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node3/.
[junit4] 2> 12190 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 12190 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 12190 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:36298_solr at time 18541797721967593
[junit4] 2> 12191 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:36298_solr] added at times [18541797721967593], now=18541797722096217
[junit4] 2> 12193 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41dfaab2b70be9T6gkzonca6dmbg13qo8ygrf36g",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18541797721967593,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541797721967593],
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 12194 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 12202 INFO (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 12207 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41dfaab2b70be9T6gkzonca6dmbg13qo8ygrf36g",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18541797721967593,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541797721967593],
[junit4] 2> "_enqueue_time_":18541797726177785,
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 12212 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger1
[junit4] 2> 12240 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 12240 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 12240 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 32 ms for event id=41dfaab2b70be9T6gkzonca6dmbg13qo8ygrf36g
[junit4] 2> 17240 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 17240 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 17240 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 17240 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 17240 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:36298_solr at time 18541802771975133
[junit4] 2> 17241 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:36298_solr] added at times [18541802771975133], now=18541802772041875
[junit4] 2> 17241 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41dfabdfb80bddT6gkzonca6dmbg13qo8ygrf36i",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18541802771975133,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541802771975133],
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 17241 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 17245 INFO (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 17245 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41dfabdfb80bddT6gkzonca6dmbg13qo8ygrf36i",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18541802771975133,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541802771975133],
[junit4] 2> "_enqueue_time_":18541802772306286,
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 17246 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518289887080000000 current time = 1518289892113000000
[junit4] 2> real diff: 5033000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 17246 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger2
[junit4] 2> 17247 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 17247 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 17247 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=41dfabdfb80bddT6gkzonca6dmbg13qo8ygrf36i
[junit4] 2> 17281 INFO (qtp1608659002-165) [n:127.0.0.1:36298_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 17290 INFO (qtp1294856939-107) [n:127.0.0.1:50598_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 17302 INFO (qtp91067448-111) [n:127.0.0.1:60585_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 17303 DEBUG (qtp1608659002-164) [n:127.0.0.1:36298_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 17303 INFO (qtp1608659002-164) [n:127.0.0.1:36298_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 17304 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 17304 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17304 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 17304 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17305 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17305 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 17305 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 17307 INFO (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 17307 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 17307 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 17316 INFO (qtp1608659002-159) [n:127.0.0.1:36298_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 17318 INFO (qtp1294856939-104) [n:127.0.0.1:50598_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 17320 INFO (qtp91067448-101) [n:127.0.0.1:60585_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 17321 DEBUG (qtp1608659002-164) [n:127.0.0.1:36298_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 17321 INFO (qtp1608659002-164) [n:127.0.0.1:36298_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 17322 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 17322 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17322 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 17322 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17323 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17323 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 17323 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:60585_solr, 127.0.0.1:36298_solr, 127.0.0.1:50598_solr]
[junit4] 2> 17323 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 17327 INFO (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 17328 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 17329 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
[junit4] 2> 17338 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1ed4e1ca{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 17339 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1758631827
[junit4] 2> 17339 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 17339 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@77b1276a: rootName = solr_36298, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@68a3012d
[junit4] 2> 17347 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 17347 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@720f4803: rootName = solr_36298, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@456706cb
[junit4] 2> 17352 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 17352 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@149e5621: rootName = solr_36298, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@759b3a9b
[junit4] 2> 17355 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36298_solr
[junit4] 2> 17357 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 17357 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 17357 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 17358 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:60585_solr) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 17359 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 17359 INFO (zkCallback-56-thread-1-processing-n:127.0.0.1:36298_solr) [n:127.0.0.1:36298_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 17363 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5e49624{/solr,null,UNAVAILABLE}
[junit4] 2> 17363 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 18309 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 18309 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:36298_solr
[junit4] 2> 18309 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:36298_solr]
[junit4] 2> 18310 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41dfac1f688f80T6gkzonca6dmbg13qo8ygrf36k",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18541803840507776,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541803840507776],
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 18310 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 18311 INFO (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 18311 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41dfac1f688f80T6gkzonca6dmbg13qo8ygrf36k",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18541803840507776,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541803840507776],
[junit4] 2> "_enqueue_time_":18541803841243630,
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 18312 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger1
[junit4] 2> 18314 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 18314 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 18314 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 18314 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 18314 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 2 ms for event id=41dfac1f688f80T6gkzonca6dmbg13qo8ygrf36k
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-2) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-3) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 23316 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:36298_solr
[junit4] 2> 23316 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:36298_solr]
[junit4] 2> 23316 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41dfad49d34a7bT6gkzonca6dmbg13qo8ygrf36m",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18541808847112827,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541808847112827],
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 23316 DEBUG (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 23315 DEBUG (ScheduledTrigger-21-thread-4) [n:127.0.0.1:50598_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 23318 INFO (ScheduledTrigger-21-thread-1) [n:127.0.0.1:50598_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 23318 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41dfad49d34a7bT6gkzonca6dmbg13qo8ygrf36m",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18541808847112827,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18541808847112827],
[junit4] 2> "_enqueue_time_":18541808847615440,
[junit4] 2> "nodeNames":["127.0.0.1:36298_solr"]}}
[junit4] 2> 23318 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518289893180000000 current time = 1518289898186000000
[junit4] 2> real diff: 5006000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 23319 INFO (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger2
[junit4] 2> 23319 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[688422546588007E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 23326 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 23326 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 23326 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 23326 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 23326 DEBUG (AutoscalingActionExecutor-22-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 7 ms for event id=41dfad49d34a7bT6gkzonca6dmbg13qo8ygrf36m
[junit4] 2> 23344 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricTrigger
[junit4] 2> 23345 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 7
[junit4] 2> 23345 DEBUG (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 23345 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 23351 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 23351 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 23351 DEBUG (OverseerAutoScalingTriggerThread-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 23356 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 23360 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 23360 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 23360 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 23361 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b973c5d{/solr,null,AVAILABLE}
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61ff9674{SSL,[ssl, http/1.1]}{127.0.0.1:56727}
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.e.j.s.Server Started @25428ms
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56727}
[junit4] 2> 23362 ERROR (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 23362 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 23363 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T19:11:38.231Z
[junit4] 2> 23371 INFO (zkConnectionManagerCallback-61-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 23372 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 23390 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45763/solr
[junit4] 2> 23392 INFO (zkConnectionManagerCallback-65-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 23398 INFO (zkConnectionManagerCallback-67-thread-1-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 23406 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 23409 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 23412 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56727_solr
[junit4] 2> 23413 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 23413 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:60585_solr) [n:127.0.0.1:60585_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 23414 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 23415 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 23539 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56727.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 23554 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56727.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 23554 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56727.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 23556 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[688422546588007E]) [n:127.0.0.1:56727_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node4/.
[junit4] 2> 23605 INFO (zkConnectionManagerCallback-71-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 23618 INFO (qtp1294856939-106) [n:127.0.0.1:50598_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=testMetricTrigger&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 23628 INFO (OverseerThreadFactory-23-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testMetricTrigger
[junit4] 2> 23756 INFO (OverseerStateUpdate-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMetricTrigger",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMetricTrigger_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50598/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 23761 INFO (OverseerStateUpdate-73272749152403461-127.0.0.1:50598_solr-n_0000000000) [n:127.0.0.1:50598_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMetricTrigger",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMetricTrigger_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:56727/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 23994 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard1_replica_n1&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 23997 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 24018 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard2_replica_n2&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 24019 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 24124 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 24124 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 24125 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 24125 INFO (zkCallback-66-thread-2-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 25078 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 25088 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 25168 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.IndexSchema [testMetricTrigger_shard1_replica_n1] Schema name=minimal
[junit4] 2> 25175 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.IndexSchema [testMetricTrigger_shard2_replica_n2] Schema name=minimal
[junit4] 2> 25216 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 25218 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 25218 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testMetricTrigger_shard1_replica_n1' using configuration from collection testMetricTrigger, trusted=true
[junit4] 2> 25217 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testMetricTrigger_shard2_replica_n2' using configuration from collection testMetricTrigger, trusted=true
[junit4] 2> 25244 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56727.solr.core.testMetricTrigger.shard2.replica_n2' (registry 'solr.core.testMetricTrigger.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 25249 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50598.solr.core.testMetricTrigger.shard1.replica_n1' (registry 'solr.core.testMetricTrigger.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52c81c6f
[junit4] 2> 25249 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 25249 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 25259 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore [[testMetricTrigger_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node1/testMetricTrigger_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node1/./testMetricTrigger_shard1_replica_n1/data/]
[junit4] 2> 25259 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore [[testMetricTrigger_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node4/testMetricTrigger_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_688422546588007E-001/tempDir-001/node4/./testMetricTrigger_shard2_replica_n2/data/]
[junit4] 2> 25798 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 25799 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 25808 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 25808 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 25813 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 25814 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 25816 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 25817 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 25832 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c572496[testMetricTrigger_shard2_replica_n2] main]
[junit4] 2> 25841 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e02fab6[testMetricTrigger_shard1_replica_n1] main]
[junit4] 2> 25847 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 25853 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 25863 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 25873 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 25880 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 25884 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 26033 INFO (searcherExecutor-42-thread-1-processing-n:127.0.0.1:50598_solr x:testMetricTrigger_shard1_replica_n1 s:shard1 c:testMetricTrigger r:core_node3) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore [testMetricTrigger_shard1_replica_n1] Registered new searcher Searcher@4e02fab6[testMetricTrigger_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 26034 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592042351128215552
[junit4] 2> 26038 INFO (searcherExecutor-41-thread-1-processing-n:127.0.0.1:56727_solr x:testMetricTrigger_shard2_replica_n2 s:shard2 c:testMetricTrigger r:core_node4) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore [testMetricTrigger_shard2_replica_n2] Registered new searcher Searcher@7c572496[testMetricTrigger_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 26038 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592042351132409856
[junit4] 2> 26061 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 26061 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 26062 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 26062 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:56727/solr/testMetricTrigger_shard2_replica_n2/
[junit4] 2> 26062 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 26062 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50598/solr/testMetricTrigger_shard1_replica_n1/
[junit4] 2> 26064 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 26064 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:56727/solr/testMetricTrigger_shard2_replica_n2/ has no replicas
[junit4] 2> 26064 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 26067 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 26067 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:50598/solr/testMetricTrigger_shard1_replica_n1/ has no replicas
[junit4] 2> 26067 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 26075 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:56727/solr/testMetricTrigger_shard2_replica_n2/ shard2
[junit4] 2> 26078 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50598/solr/testMetricTrigger_shard1_replica_n1/ shard1
[junit4] 2> 26180 INFO (zkCallback-66-thread-2-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26180 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26181 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26181 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26183 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 26186 INFO (qtp1294856939-108) [n:127.0.0.1:50598_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard1_replica_n1&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2195
[junit4] 2> 26227 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 26230 INFO (qtp1680592517-192) [n:127.0.0.1:56727_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard2_replica_n2&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2212
[junit4] 2> 26237 INFO (qtp1294856939-106) [n:127.0.0.1:50598_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 26331 INFO (zkCallback-66-thread-2-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26331 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:56727_solr) [n:127.0.0.1:56727_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26332 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 26333 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:50598_solr) [n:127.0.0.1:50598_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
[...truncated too long message...]
r.java:1379)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService.execute(InstrumentedExecutorService.java:61)
[junit4] 2> at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.submit(HttpShardHandler.java:203)
[junit4] 2> at org.apache.solr.update.PeerSync.requestFingerprint(PeerSync.java:425)
[junit4] 2> at org.apache.solr.update.PeerSync.alreadyInSync(PeerSync.java:384)
[junit4] 2> at org.apache.solr.update.PeerSync.sync(PeerSync.java:253)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:593)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:310)
[junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:294)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.RecoveryStrategy Replay not started, or was not successful... still buffering updates.
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.RecoveryStrategy RecoveryStrategy has been closed
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[false]
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.SolrCore [utilizenodecoll_shard2_replica_n9] CLOSING SolrCore org.apache.solr.core.SolrCore@54e4987f
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.utilizenodecoll.shard2.replica_n9, tag=1424267391
[junit4] 2> 1051462 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3adfac0d: rootName = solr_40822, domain = solr.core.utilizenodecoll.shard2.replica_n9, service url = null, agent id = null] for registry solr.core.utilizenodecoll.shard2.replica_n9 / com.codahale.metrics.MetricRegistry@72e212af
[junit4] 2> 1051470 INFO (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.utilizenodecoll.shard2.leader, tag=1424267391
[junit4] 2> 1051473 WARN (recoveryExecutor-1916-thread-1-processing-n:127.0.0.1:40822_solr x:utilizenodecoll_shard2_replica_n9 s:shard2 c:utilizenodecoll r:core_node10) [n:127.0.0.1:40822_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.RecoveryStrategy Stopping recovery for core=[utilizenodecoll_shard2_replica_n9] coreNodeName=[core_node10]
[junit4] 2> 1051495 INFO (jetty-closer-1861-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f93c880{/solr,null,UNAVAILABLE}
[junit4] 2> 1051495 INFO (jetty-closer-1861-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1051496 ERROR (SUITE-TestUtilizeNode-seed#[688422546588007E]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1051496 INFO (SUITE-TestUtilizeNode-seed#[688422546588007E]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53597 53597
[junit4] 2> 1051583 INFO (Thread-3606) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53597 53597
[junit4] 2> 1051583 WARN (Thread-3606) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/utilizenodecoll/terms/shard2
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2> 5 /solr/collections/utilizenodecoll/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 54 /solr/collections/utilizenodecoll/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/73272816889757705-127.0.0.1:33400_solr-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73272816889757708-127.0.0.1:41180_solr-n_0000000000
[junit4] 2> 2 /solr/collections/utilizenodecoll/leader_elect/shard2/election/73272816889757706-core_node8-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/live_nodes
[junit4] 2> 9 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestUtilizeNode_688422546588007E-001
[junit4] 2> Feb 10, 2018 7:28:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=sr-BA, timezone=America/Porto_Velho
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=40206736,total=523763712
[junit4] 2> NOTE: All tests run in this JVM: [ForceLeaderTest, TestClusterStateProvider, TestNumericRangeQuery64, SortSpecParsingTest, HdfsRecoverLeaseTest, TestRuleBasedAuthorizationPlugin, TermVectorComponentTest, TestJmxIntegration, TestQueryTypes, ClusterStateUpdateTest, StatsReloadRaceTest, ReturnFieldsTest, TestCoreBackup, TestBadConfig, TestRangeQuery, SolrCmdDistributorTest, DistributedDebugComponentTest, DistributedFacetSimpleRefinementLongTailTest, TestImpersonationWithHadoopAuth, AutoscalingHistoryHandlerTest, TestSchemaManager, TestSchemaVersionResource, SearchHandlerTest, TestCloudPseudoReturnFields, SearchHandlerTest, TestCodecSupport, SpellCheckCollatorWithCollapseTest, TestLeaderElectionWithEmptyReplica, SolrMetricManagerTest, TestTolerantSearch, TestReloadAndDeleteDocs, SolrXmlInZkTest, SpellPossibilityIteratorTest, TestLFUCache, SolrShardReporterTest, SolrJmxReporterTest, TestSubQueryTransformerDistrib, AutoScalingHandlerTest, SchemaApiFailureTest, TestNamedUpdateProcessors, TestSystemCollAutoCreate, TestLockTree, TestDistribDocBasedVersion, HDFSCollectionsAPITest, TestRestManager, CopyFieldTest, InfixSuggestersTest, DeleteReplicaTest, MetricUtilsTest, BlobRepositoryCloudTest, TestRawTransformer, HdfsSyncSliceTest, MigrateRouteKeyTest, RegexBoostProcessorTest, DistributedFacetPivotLargeTest, TestExportWriter, TestDistributedMissingSort, TestCollationFieldDocValues, LIRRollingUpdatesTest, TestRandomFlRTGCloud, TestLegacyField, ConfigureRecoveryStrategyTest, UniqFieldsUpdateProcessorFactoryTest, TestQuerySenderListener, TestSolrCoreProperties, RollingRestartTest, CollectionsAPISolrJTest, TestCoreDiscovery, TestConfigSetProperties, HdfsAutoAddReplicasIntegrationTest, TestSolrDeletionPolicy2, TestSuggestSpellingConverter, TestAnalyzedSuggestions, TestFieldSortValues, ConnectionReuseTest, DistributedQueryComponentOptimizationTest, CacheHeaderTest, PreAnalyzedUpdateProcessorTest, WordBreakSolrSpellCheckerTest, SmileWriterTest, DocExpirationUpdateProcessorFactoryTest, UpdateLogTest, TestHdfsUpdateLog, UninvertDocValuesMergePolicyTest, TestLegacyFieldCache, CoreMergeIndexesAdminHandlerTest, TestCustomStream, DateMathParserTest, TestJavabinTupleStreamParser, TestMaxScoreQueryParser, TestUtilizeNode]
[junit4] Completed [298/775 (2!)] on J2 in 8.92s, 1 test, 1 failure <<< FAILURES!
[...truncated 49815 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 365 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/365/
5 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235:E6D4FF36520FE7C7]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown(TriggerIntegrationTest.java:1177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235:6D66A55D734D447A]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235]:0)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235:BA96362F96ED6D32]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12223 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/init-core-data-001
[junit4] 2> 117865 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=13 numCloses=13
[junit4] 2> 117865 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 117866 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 117867 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 117867 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001
[junit4] 2> 117867 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 117867 INFO (Thread-172) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 117867 INFO (Thread-172) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 117870 ERROR (Thread-172) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 117967 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44961
[junit4] 2> 117969 INFO (zkConnectionManagerCallback-222-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 117975 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T10:27:37-11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 117981 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T10:27:37-11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 117981 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T10:27:37-11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 117985 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 117985 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 117986 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 117986 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 117986 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 117986 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 117986 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19a832d4{/solr,null,AVAILABLE}
[junit4] 2> 117987 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 117987 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 117987 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 117993 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18f213bc{/solr,null,AVAILABLE}
[junit4] 2> 117994 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2280e293{/solr,null,AVAILABLE}
[junit4] 2> 117996 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5111a7ba{SSL,[ssl, http/1.1]}{127.0.0.1:49193}
[junit4] 2> 117996 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3707502f{SSL,[ssl, http/1.1]}{127.0.0.1:40767}
[junit4] 2> 117996 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@72dad314{SSL,[ssl, http/1.1]}{127.0.0.1:39908}
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.e.j.s.Server Started @119956ms
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-1) [ ] o.e.j.s.Server Started @119956ms
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39908}
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40767}
[junit4] 2> 117997 ERROR (jetty-launcher-219-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 117997 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T14:25:11.649Z
[junit4] 2> 117998 ERROR (jetty-launcher-219-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-3) [ ] o.e.j.s.Server Started @119957ms
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T14:25:11.650Z
[junit4] 2> 117998 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49193}
[junit4] 2> 118001 ERROR (jetty-launcher-219-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 118001 INFO (zkConnectionManagerCallback-224-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118001 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 118001 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 118001 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 118001 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T14:25:11.653Z
[junit4] 2> 118003 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 118005 INFO (zkConnectionManagerCallback-228-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118013 INFO (zkConnectionManagerCallback-226-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118015 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 118015 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 118020 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10450274d670003, likely client has closed socket
[junit4] 2> 118021 INFO (jetty-launcher-219-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44961/solr
[junit4] 2> 118026 INFO (jetty-launcher-219-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44961/solr
[junit4] 2> 118033 INFO (jetty-launcher-219-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44961/solr
[junit4] 2> 118037 INFO (zkConnectionManagerCallback-232-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118043 INFO (zkConnectionManagerCallback-240-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118043 INFO (zkConnectionManagerCallback-236-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118050 INFO (zkConnectionManagerCallback-242-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118054 INFO (zkConnectionManagerCallback-244-thread-1-processing-n:127.0.0.1:39908_solr) [n:127.0.0.1:39908_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118057 INFO (zkConnectionManagerCallback-246-thread-1-processing-n:127.0.0.1:49193_solr) [n:127.0.0.1:49193_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118089 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 118093 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40767_solr
[junit4] 2> 118094 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer Overseer (id=73271623677313031-127.0.0.1:40767_solr-n_0000000000) starting
[junit4] 2> 118111 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 118121 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40767_solr
[junit4] 2> 118121 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 118123 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 118124 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
[junit4] 2> 118124 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 118124 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 118126 INFO (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 118157 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 118161 DEBUG (OverseerAutoScalingTriggerThread-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 118161 DEBUG (ScheduledTrigger-391-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 118171 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 118173 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 118173 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 118175 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39908_solr
[junit4] 2> 118175 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:39908_solr"} current state version: 0
[junit4] 2> 118175 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39908_solr
[junit4] 2> 118176 INFO (zkCallback-241-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 118181 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 118182 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49193_solr
[junit4] 2> 118183 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:49193_solr"} current state version: 0
[junit4] 2> 118183 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:49193_solr
[junit4] 2> 118186 INFO (zkCallback-241-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 118229 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:39908_solr) [n:127.0.0.1:39908_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 118232 INFO (zkCallback-245-thread-1-processing-n:127.0.0.1:49193_solr) [n:127.0.0.1:49193_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 118296 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40767.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118305 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40767.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118305 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40767.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118306 INFO (jetty-launcher-219-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/.
[junit4] 2> 118333 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49193.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118334 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39908.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118342 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39908.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118343 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39908.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118344 INFO (jetty-launcher-219-thread-2) [n:127.0.0.1:39908_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/.
[junit4] 2> 118345 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49193.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118346 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49193.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f98d4c6
[junit4] 2> 118347 INFO (jetty-launcher-219-thread-3) [n:127.0.0.1:49193_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/.
[junit4] 2> 118398 INFO (zkConnectionManagerCallback-254-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118399 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10450274d67000a, likely client has closed socket
[junit4] 2> 118402 INFO (zkConnectionManagerCallback-258-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 118402 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 118403 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[D76A92D22CA59235]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44961/solr ready
[junit4] 2> 118427 INFO (qtp249234219-942) [n:127.0.0.1:39908_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 118428 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 118429 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 118430 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 118431 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 118433 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 118434 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 118435 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 118436 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 118438 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 118439 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 118440 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 118441 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 118442 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 118443 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 118445 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 118446 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 118447 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 118448 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 118449 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 118451 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 118452 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 118453 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 118454 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 118455 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 118457 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 118458 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 118459 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 118460 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 118461 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 118463 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 118464 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 118470 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 118473 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 118478 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 118479 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 118480 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 118482 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 118483 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 118484 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 118486 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 118487 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 118488 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 118490 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 118491 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 118492 INFO (OverseerThreadFactory-393-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 118513 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 118513 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 118516 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 118616 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 118616 INFO (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 118618 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 118618 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 118618 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 118618 INFO (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 118619 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 118619 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 118620 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 118620 INFO (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 118621 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 118621 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 118721 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 118850 INFO (qtp249234219-928) [n:127.0.0.1:39908_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 118850 INFO (qtp249234219-928) [n:127.0.0.1:39908_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 118853 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "node_name":"127.0.0.1:39908_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 118853 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "node_name":"127.0.0.1:39908_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 118853 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:39908/solr","node_name":"127.0.0.1:39908_solr","state":"down","type":"NRT"}
[junit4] 2> 118853 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "node_name":"127.0.0.1:39908_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 118863 INFO (qtp1565528396-924) [n:127.0.0.1:40767_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 118864 INFO (qtp1565528396-924) [n:127.0.0.1:40767_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 118874 INFO (qtp1895696977-932) [n:127.0.0.1:49193_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 118874 INFO (qtp1895696977-932) [n:127.0.0.1:49193_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 118875 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "node_name":"127.0.0.1:40767_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 118875 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "node_name":"127.0.0.1:40767_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 118875 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:40767/solr","node_name":"127.0.0.1:40767_solr","state":"down","type":"NRT"}
[junit4] 2> 118875 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "node_name":"127.0.0.1:40767_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "node_name":"127.0.0.1:39908_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 118877 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "node_name":"127.0.0.1:49193_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 118877 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "node_name":"127.0.0.1:49193_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 118877 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:49193/solr","node_name":"127.0.0.1:49193_solr","state":"down","type":"NRT"}
[junit4] 2> 118877 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:49193/solr",
[junit4] 2> "node_name":"127.0.0.1:49193_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:40767/solr",
[junit4] 2> "node_name":"127.0.0.1:40767_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:39908/solr",
[junit4] 2> "node_name":"127.0.0.1:39908_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 118978 DEBUG (OverseerStateUpdate-73271623677313031-127.0.0.1:40767_solr-n_0000000000) [n:127.0.0.1:40767_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 118979 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:39908_solr) [n:127.0.0.1:39908_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 118979 INFO (zkCallback-245-thread-1-processing-n:127.0.0.1:49193_solr) [n:127.0.0.1:49193_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 118979 INFO (zkCallback-241-thread-1-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 118981 INFO (zkCallback-243-thread-2-processing-n:127.0.0.1:39908_solr) [n:127.0.0.1:39908_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 118981 INFO (zkCallback-245-thread-2-processing-n:127.0.0.1:49193_solr) [n:127.0.0.1:49193_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 118981 INFO (zkCallback-241-thread-2-processing-n:127.0.0.1:40767_solr) [n:127.0.0.1:40767_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 119162 DEBUG (ScheduledTrigger-391-thread-1) [n:127.0.0.1:40767_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 119868 INFO (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 119869 INFO (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 119870 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
[junit4] 2> 119871 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 119871 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
[junit4] 2> 119871 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 119872 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
[junit4] 2> 119872 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 119872 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib
[junit4] 2> 119872 WARN (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 119878 INFO (qtp249234219-928) [n:127.0.0.1:39908_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 119886 INFO (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 119886 INFO (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 119886 WARN (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 119890 INFO (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 119890 INFO (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 119891 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 119891 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 119891 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 119891 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 119892 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 119892 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 119892 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
[junit4] 2> 119892 WARN (qtp1565528396-924) [n:127.0.0.1:40767_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 119893 WARN (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 119893 WARN (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_D76A92D22CA59235-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 119893 WARN (qtp1895696977-932) [n:127.0.0.1:49193_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenk
[...truncated too long message...]
.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> veebr 10, 2018 7:01:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=30537, name=TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[D76A92D22CA59235], state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
[junit4] 2> 9506027 INFO (TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[D76A92D22CA59235]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueue
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestGenericDistributedQueue_D76A92D22CA59235-001
[junit4] 2> Feb 10, 2018 5:01:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=4, maxDocsPerChunk=81, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=4, blockSize=5)), sim=RandomSimilarity(queryNorm=false): {}, locale=et-EE, timezone=Libya
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=235010664,total=515899392
[junit4] 2> NOTE: All tests run in this JVM: [TestSortByMinMaxFunction, DocValuesMissingTest, TestWriterPerf, TestShardHandlerFactory, ConcurrentCreateRoutedAliasTest, DistributedVersionInfoTest, TestSolrQueryParser, TestCloudSchemaless, TestStressUserVersions, NumberUtilsTest, HighlighterConfigTest, DefaultValueUpdateProcessorTest, TestCustomDocTransformer, TestSolrXml, BasicDistributedZkTest, SegmentsInfoRequestHandlerTest, MoveReplicaTest, TestPolicyCloud, TestLuceneMatchVersion, TestFieldCollectionResource, RequestHandlersTest, TestIndexSearcher, SolrCoreMetricManagerTest, ZkSolrClientTest, TestCryptoKeys, TestConfigReload, HdfsBasicDistributedZk2Test, TestFieldCacheSort, DistributedFacetPivotSmallAdvancedTest, DistributedTermsComponentTest, TestComponentsName, DistribDocExpirationUpdateProcessorTest, TestGroupingSearch, TestConfigSets, TestSortableTextField, AutoAddReplicasIntegrationTest, RAMDirectoryFactoryTest, TestLegacyNumericUtils, StatsComponentTest, TestXmlQParserPlugin, SuggesterWFSTTest, HttpPartitionOnCommitTest, TestCollectionAPI, TestMultiWordSynonyms, SolrCLIZkUtilsTest, ChaosMonkeyNothingIsSafeTest, TestUniqueKeyFieldResource, TestSimpleQParserPlugin, TestRandomCollapseQParserPlugin, CoreSorterTest, CursorMarkTest, SpatialRPTFieldTypeTest, RankQueryTest, TestZkAclsWithHadoopAuth, TestStreamBody, MetricsHandlerTest, TestCrossCoreJoin, TestDownShardTolerantSearch, TestSolrConfigHandlerConcurrent, SolrCloudReportersTest, TestBinaryField, ConjunctionSolrSpellCheckerTest, BigEndianAscendingWordSerializerTest, EchoParamsTest, TestDefaultStatsCache, CoreAdminRequestStatusTest, TestConfig, PreAnalyzedFieldManagedSchemaCloudTest, CheckHdfsIndexTest, TestFastOutputStream, V2StandaloneTest, TestCloudNestedDocsSort, SolrIndexSplitterTest, CursorPagingTest, TestStressInPlaceUpdates, TestRawResponseWriter, TestMiniSolrCloudClusterSSL, TestGraphTermsQParserPlugin, TestDocumentBuilder, ShardRoutingTest, TestLocalFSCloudBackupRestore, TestSchemaSimilarityResource, ReplicationFactorTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestManagedSynonymGraphFilterFactory, PluginInfoTest, CdcrBootstrapTest, SparseHLLTest, HttpTriggerListenerTest, ActionThrottleTest, HdfsChaosMonkeySafeLeaderTest, OpenCloseCoreStressTest, TestSlowCompositeReaderWrapper, HdfsWriteToMultipleCollectionsTest, TestOmitPositions, SolrPluginUtilsTest, TestPseudoReturnFields, CreateCollectionCleanupTest, TestRTimerTree, CollectionsAPIDistributedZkTest, TestDocTermOrds, TestConfigSetsAPI, TestPayloadCheckQParserPlugin, TestCollectionAPIs, RequestLoggingTest, NodeLostTriggerTest, OpenExchangeRatesOrgProviderTest, DeleteShardTest, CloneFieldUpdateProcessorFactoryTest, BadIndexSchemaTest, SolrRequestParserTest, JavabinLoaderTest, TestCloudRecovery, ZkCLITest, RuleEngineTest, HdfsDirectoryFactoryTest, OutputWriterTest, SolrCoreCheckLockOnStartupTest, AnalysisErrorHandlingTest, AssignBackwardCompatibilityTest, TestSort, TestIndexingPerformance, CollectionReloadTest, TestInPlaceUpdatesStandalone, ExternalFileFieldSortTest, TestCSVResponseWriter, TestZkChroot, TestDistributedSearch, TestHashPartitioner, TestJoin, BasicFunctionalityTest, DirectUpdateHandlerTest, HighlighterTest, DistributedQueryElevationComponentTest, CoreAdminHandlerTest, SuggesterTest, SpatialFilterTest, SchemaVersionSpecificBehaviorTest, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, SolrInfoBeanTest, IndexBasedSpellCheckerTest, TestQueryUtils, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, DocumentBuilderTest, FieldAnalysisRequestHandlerTest, LoggingHandlerTest, IndexSchemaTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, SearchHandlerTest, TestQuerySenderNoQuery, SolrIndexConfigTest, TestStressRecovery, TestFastWriter, DateMathParserTest, DateFieldTest, DOMUtilTest, TestSolrJ, TestLRUCache, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, SolrTestCaseJ4Test, TestHighlightDedupGrouping, TestLuceneIndexBackCompat, AsyncCallRequestStatusResponseTest, ChaosMonkeySafeLeaderWithPullReplicasTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteStatusTest, DocValuesNotIndexedTest, LeaderElectionContextKeyTest, LeaderInitiatedRecoveryOnShardRestartTest, LegacyCloudClusterPropTest, MissingSegmentRecoveryTest, MoveReplicaHDFSFailoverTest, RemoteQueryErrorTest, SharedFSAutoReplicaFailoverTest, SolrCloudExampleTest, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestConfigSetsAPIZkFailure, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLeaderInitiatedRecoveryThread, TestPrepRecovery, TestPullReplicaErrorHandling, TestSSLRandomization, TestSizeLimitedDistributedMap, TestSkipOverseerOperations, TestStressLiveNodes, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TestUtilizeNode, TlogReplayBufferedWhileIndexingTest, ZkFailoverTest, HdfsCollectionsAPIDistributedZkTest, TestCollectionsAPIViaSolrCloudCluster, TestHdfsCloudBackupRestore, ComputePlanActionTest, SystemLogListenerTest, TestClusterStateProvider, TestComputePlanAction, TestDistribStateManager, TestGenericDistributedQueue]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.seed=D76A92D22CA59235 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Libya -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestGenericDistributedQueue (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([D76A92D22CA59235]:0)
[junit4] Completed [774/774 (3!)] on J0 in 7220.45s, 1 test, 2 errors <<< FAILURES!
[...truncated 15615 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:846: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:117: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2329: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2087: Rat problems were found!
Total time: 188 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 364 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/364/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([7624FAE60B613F35:BF91B8480206F9C0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:713)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([7624FAE60B613F35:1BD85E1BB129C032]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12460 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/init-core-data-001
[junit4] 2> 250149 WARN (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=146 numCloses=146
[junit4] 2> 250149 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 250150 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 250150 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 250151 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001
[junit4] 2> 250151 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 250151 INFO (Thread-719) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 250151 INFO (Thread-719) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 250152 ERROR (Thread-719) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 250251 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:60323
[junit4] 2> 250268 INFO (zkConnectionManagerCallback-715-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250273 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 250280 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 250284 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 250284 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 250284 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 250285 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c07dde4{/solr,null,AVAILABLE}
[junit4] 2> 250287 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@622fd3ca{HTTP/1.1,[http/1.1]}{127.0.0.1:36809}
[junit4] 2> 250287 INFO (jetty-launcher-712-thread-2) [ ] o.e.j.s.Server Started @252928ms
[junit4] 2> 250287 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36809}
[junit4] 2> 250293 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 250293 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 250294 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 250294 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f4babe1{/solr,null,AVAILABLE}
[junit4] 2> 250294 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5685c0ef{HTTP/1.1,[http/1.1]}{127.0.0.1:40536}
[junit4] 2> 250294 INFO (jetty-launcher-712-thread-1) [ ] o.e.j.s.Server Started @252935ms
[junit4] 2> 250294 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40536}
[junit4] 2> 250295 ERROR (jetty-launcher-712-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 250295 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 250295 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 250295 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 250295 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T12:56:44.556Z
[junit4] 2> 250296 INFO (zkConnectionManagerCallback-717-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250297 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 250306 ERROR (jetty-launcher-712-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 250306 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 250306 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 250306 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 250306 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T12:56:44.567Z
[junit4] 2> 250332 INFO (jetty-launcher-712-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60323/solr
[junit4] 2> 250339 INFO (zkConnectionManagerCallback-719-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250339 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 250360 INFO (zkConnectionManagerCallback-723-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250392 INFO (jetty-launcher-712-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60323/solr
[junit4] 2> 250404 INFO (zkConnectionManagerCallback-726-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250424 INFO (zkConnectionManagerCallback-729-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250424 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044fd652840005, likely client has closed socket
[junit4] 2> 250448 INFO (zkConnectionManagerCallback-732-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250458 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 250458 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 250459 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36809_solr
[junit4] 2> 250459 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40536_solr
[junit4] 2> 250459 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.Overseer Overseer (id=73271275870748678-127.0.0.1:36809_solr-n_0000000000) starting
[junit4] 2> 250465 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 250471 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 250500 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36809_solr
[junit4] 2> 250500 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 250506 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 250506 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 250508 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 250508 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:36809_solr, 127.0.0.1:40536_solr]
[junit4] 2> 250508 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 250508 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 250512 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 250512 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 250512 DEBUG (ScheduledTrigger-1442-thread-1) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 250591 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36809.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250598 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36809.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250598 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40536.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250598 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36809.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250599 INFO (jetty-launcher-712-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node2/.
[junit4] 2> 250605 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40536.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250605 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40536.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250606 INFO (jetty-launcher-712-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node1/.
[junit4] 2> 250649 INFO (zkConnectionManagerCallback-737-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250652 INFO (zkConnectionManagerCallback-741-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250653 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 250654 INFO (SUITE-TriggerIntegrationTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60323/solr ready
[junit4] 2> 250669 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 250669 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 250669 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 250669 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 250671 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 250671 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 250672 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 250684 INFO (qtp989304898-3112) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 250686 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 250686 DEBUG (qtp989304898-3104) [n:127.0.0.1:36809_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 250687 INFO (qtp989304898-3104) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 250687 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 250689 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:36809_solr, 127.0.0.1:40536_solr]
[junit4] 2> 250689 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 250690 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 250691 INFO (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 250691 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 250692 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 250692 DEBUG (ScheduledTrigger-1442-thread-1) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 250692 DEBUG (ScheduledTrigger-1442-thread-1) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 250692 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 250700 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 250700 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 250700 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29a00abf{/solr,null,AVAILABLE}
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@171ac14c{HTTP/1.1,[http/1.1]}{127.0.0.1:59749}
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.e.j.s.Server Started @253342ms
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59749}
[junit4] 2> 250701 ERROR (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 250701 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T12:56:44.962Z
[junit4] 2> 250709 INFO (zkConnectionManagerCallback-743-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250710 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 250717 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60323/solr
[junit4] 2> 250720 INFO (zkConnectionManagerCallback-747-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250728 INFO (zkConnectionManagerCallback-749-thread-1-processing-n:127.0.0.1:59749_solr) [n:127.0.0.1:59749_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 250734 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 250735 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 250737 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59749_solr
[junit4] 2> 250744 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 250745 INFO (zkCallback-740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 250745 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 250752 INFO (zkCallback-748-thread-1-processing-n:127.0.0.1:59749_solr) [n:127.0.0.1:59749_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 250794 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59749.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59749.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59749.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 250803 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [n:127.0.0.1:59749_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node3/.
[junit4] 2> 251693 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 251693 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 251694 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:59749_solr at time 18519316617994054
[junit4] 2> 252694 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 252694 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 253695 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 253695 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 253695 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:59749_solr] added at times [18519316617994054], now=18519318619384759
[junit4] 2> 253695 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41cb38687f9f46Tegwomox6y42qctmfdjryt5ohk",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18519316617994054,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18519316617994054],
[junit4] 2> "nodeNames":["127.0.0.1:59749_solr"]}}
[junit4] 2> 253696 DEBUG (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
[junit4] 2> 253697 INFO (ScheduledTrigger-1442-thread-2) [n:127.0.0.1:36809_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 253699 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41cb38687f9f46Tegwomox6y42qctmfdjryt5ohk",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18519316617994054,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18519316617994054],
[junit4] 2> "_enqueue_time_":18519318620713764,
[junit4] 2> "nodeNames":["127.0.0.1:59749_solr"]}}
[junit4] 2> 253709 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
[junit4] 2> 253709 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 9 ms for event id=41cb38687f9f46Tegwomox6y42qctmfdjryt5ohk
[junit4] 2> 253710 INFO (qtp989304898-3112) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 253711 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 253713 INFO (qtp128196007-3172) [n:127.0.0.1:59749_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 253713 DEBUG (qtp128196007-3168) [n:127.0.0.1:59749_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 253716 INFO (qtp128196007-3168) [n:127.0.0.1:59749_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=14
[junit4] 2> 253716 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 253716 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:59749_solr, 127.0.0.1:36809_solr, 127.0.0.1:40536_solr]
[junit4] 2> 253716 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 253716 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 253717 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 253717 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 255716 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 255733 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSetProperties
[junit4] 2> 255734 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 255734 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 255734 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 255735 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 255735 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 255735 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 256738 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerScheduleDelaySeconds from: 1 to: 4
[junit4] 2> 260742 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerScheduleDelaySeconds from: 4 to: 1
[junit4] 2> 270750 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerCorePoolSize from: 4 to: 6
[junit4] 2> 280755 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSetProperties
[junit4] 2> 280777 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 280777 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 280778 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 280778 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 280778 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 280778 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 280778 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 280781 INFO (qtp989304898-3108) [n:127.0.0.1:36809_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 280790 INFO (qtp989304898-3108) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 280797 INFO (qtp989304898-3109) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 280799 INFO (qtp1419331519-3115) [n:127.0.0.1:40536_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 280800 INFO (qtp128196007-3174) [n:127.0.0.1:59749_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 280800 DEBUG (qtp989304898-3112) [n:127.0.0.1:36809_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 280801 INFO (qtp989304898-3112) [n:127.0.0.1:36809_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 280801 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 280801 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventQueueAction instantiated
[junit4] 2> 280801 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:59749_solr, 127.0.0.1:36809_solr, 127.0.0.1:40536_solr]
[junit4] 2> 280801 DEBUG (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventQueueAction}], enabled=true}
[junit4] 2> 280801 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 280802 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 280802 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 280802 DEBUG (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 280803 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 280803 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 280803 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 280804 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 280804 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 280804 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59b27563{/solr,null,AVAILABLE}
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7517f696{HTTP/1.1,[http/1.1]}{127.0.0.1:59138}
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.Server Started @283446ms
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59138}
[junit4] 2> 280805 ERROR (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 280805 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T12:57:15.066Z
[junit4] 2> 280807 INFO (zkConnectionManagerCallback-753-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 280807 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 280815 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60323/solr
[junit4] 2> 280817 INFO (zkConnectionManagerCallback-757-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 280821 INFO (zkConnectionManagerCallback-759-thread-1-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 280827 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 280829 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 280831 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59138_solr
[junit4] 2> 280836 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 280836 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 280836 INFO (zkCallback-740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 280836 INFO (zkCallback-748-thread-1-processing-n:127.0.0.1:59749_solr) [n:127.0.0.1:59749_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 280837 INFO (zkCallback-758-thread-1-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 280880 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59138.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 280888 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59138.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 280888 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59138.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 280889 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [n:127.0.0.1:59138_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node4/.
[junit4] 2> 281804 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 281804 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 281804 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:59138_solr at time 18519346728468505
[junit4] 2> 282787 INFO (OverseerCollectionConfigSetProcessor-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 282805 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 282805 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 282805 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ firing registered processor for nodes: [127.0.0.1:59138_solr] added at times [18519346728468505], now=18519347729322395
[junit4] 2> 282805 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41cb3f6b390019Tegwomox6y42qctmfdjryt5ohm",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":18519346728468505,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18519346728468505],
[junit4] 2> "nodeNames":["127.0.0.1:59138_solr"]}}
[junit4] 2> 282805 DEBUG (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerEQ]
[junit4] 2> 282806 INFO (ScheduledTrigger-1442-thread-3) [n:127.0.0.1:36809_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 282806 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41cb3f6b390019Tegwomox6y42qctmfdjryt5ohm",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":18519346728468505,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18519346728468505],
[junit4] 2> "_enqueue_time_":18519347729522282,
[junit4] 2> "nodeNames":["127.0.0.1:59138_solr"]}}
[junit4] 2> 282806 INFO (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.TriggerIntegrationTest -- event: {
[junit4] 2> "id":"41cb3f6b390019Tegwomox6y42qctmfdjryt5ohm",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":18519346728468505,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18519346728468505],
[junit4] 2> "_enqueue_time_":18519347729522282,
[junit4] 2> "nodeNames":["127.0.0.1:59138_solr"]}}
[junit4] 2> 282807 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@622fd3ca{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 282808 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1644809303
[junit4] 2> 282808 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 282808 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@73e1b6c9: rootName = solr_36809, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@8f83548
[junit4] 2> 282809 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerEQ after 5000ms
[junit4] 2> 282809 DEBUG (AutoscalingActionExecutor-1443-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 2 ms for event id=41cb3f6b390019Tegwomox6y42qctmfdjryt5ohm
[junit4] 2> 282814 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 282814 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20a5f128: rootName = solr_36809, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@19e24f9
[junit4] 2> 282818 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 282818 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6b0cc398: rootName = solr_36809, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@15c1e694
[junit4] 2> 282819 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36809_solr
[junit4] 2> 282820 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 282820 INFO (zkCallback-748-thread-1-processing-n:127.0.0.1:59749_solr) [n:127.0.0.1:59749_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 282820 INFO (zkCallback-740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 282820 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 282820 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:36809_solr) [n:127.0.0.1:36809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 282824 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 282824 INFO (zkCallback-758-thread-1-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 282824 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.Overseer Overseer (id=73271275870748678-127.0.0.1:36809_solr-n_0000000000) closing
[junit4] 2> 282824 INFO (OverseerStateUpdate-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36809_solr
[junit4] 2> 282825 WARN (OverseerAutoScalingTriggerThread-73271275870748678-127.0.0.1:36809_solr-n_0000000000) [n:127.0.0.1:36809_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 282828 DEBUG (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 282829 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40536_solr
[junit4] 2> 282829 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6c07dde4{/solr,null,UNAVAILABLE}
[junit4] 2> 282830 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 282830 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.Overseer Overseer (id=73271275870748676-127.0.0.1:40536_solr-n_0000000001) starting
[junit4] 2> 282836 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 282837 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 282837 INFO (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventQueueAction instantiated
[junit4] 2> 282837 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:59749_solr, 127.0.0.1:40536_solr, 127.0.0.1:59138_solr]
[junit4] 2> 282837 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventQueueAction}], enabled=true}
[junit4] 2> 282837 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:59749_solr, 127.0.0.1:40536_solr, 127.0.0.1:59138_solr]
[junit4] 2> 282838 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion -1
[junit4] 2> 282838 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 282840 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 282840 DEBUG (ScheduledTrigger-1465-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 282840 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 282841 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 282841 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 283841 DEBUG (ScheduledTrigger-1465-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 283842 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 283842 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 284842 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 284843 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 284843 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 285842 DEBUG (ScheduledTrigger-1465-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 285843 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 285843 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 286842 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 286843 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 286843 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 287842 DEBUG (ScheduledTrigger-1465-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 287843 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 287843 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 288843 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 288844 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 288844 DEBUG (ScheduledTrigger-1465-thread-3) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 289843 DEBUG (ScheduledTrigger-1465-thread-1) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 289844 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 289844 DEBUG (ScheduledTrigger-1465-thread-2) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 290831 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TriggerIntegrationTest -Dtests.method=testEventQueue -Dtests.seed=7624FAE60B613F35 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=fr-CH -Dtests.timezone=SystemV/CST6CDT -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 10.1s J1 | TriggerIntegrationTest.testEventQueue <<<
[junit4] > Throwable #1: java.lang.AssertionError: action wasn't interrupted
[junit4] > at __randomizedtesting.SeedInfo.seed([7624FAE60B613F35:BF91B8480206F9C0]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:713)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 290844 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 290844 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 290844 DEBUG (ScheduledTrigger-1465-thread-4) [n:127.0.0.1:40536_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 290859 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricTrigger
[junit4] 2> 290860 INFO (TEST-TriggerIntegrationTest.testMetricTrigger-seed#[7624FAE60B613F35]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 9
[junit4] 2> 290860 DEBUG (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 290860 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 290861 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 290862 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 290862 DEBUG (OverseerAutoScalingTriggerThread-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 290863 INFO (zkConnectionManagerCallback-763-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 290866 INFO (qtp1419331519-3115) [n:127.0.0.1:40536_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=testMetricTrigger&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 290873 INFO (OverseerThreadFactory-1467-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testMetricTrigger
[junit4] 2> 290980 INFO (OverseerStateUpdate-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMetricTrigger",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMetricTrigger_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59138/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 290983 INFO (OverseerStateUpdate-73271275870748676-127.0.0.1:40536_solr-n_0000000001) [n:127.0.0.1:40536_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMetricTrigger",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMetricTrigger_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40536/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 291187 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard2_replica_n2&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 291187 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard1_replica_n1&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 291189 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 291189 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 291294 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 291294 INFO (zkCallback-758-thread-1-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 291294 INFO (zkCallback-758-thread-2-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 291294 INFO (zkCallback-725-thread-2-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 292211 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 292211 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 292217 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.IndexSchema [testMetricTrigger_shard2_replica_n2] Schema name=minimal
[junit4] 2> 292218 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.IndexSchema [testMetricTrigger_shard1_replica_n1] Schema name=minimal
[junit4] 2> 292220 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 292220 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testMetricTrigger_shard2_replica_n2' using configuration from collection testMetricTrigger, trusted=true
[junit4] 2> 292220 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 292220 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testMetricTrigger_shard1_replica_n1' using configuration from collection testMetricTrigger, trusted=true
[junit4] 2> 292220 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40536.solr.core.testMetricTrigger.shard2.replica_n2' (registry 'solr.core.testMetricTrigger.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 292220 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59138.solr.core.testMetricTrigger.shard1.replica_n1' (registry 'solr.core.testMetricTrigger.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c8589ec
[junit4] 2> 292220 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 292221 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore [[testMetricTrigger_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node1/testMetricTrigger_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node1/./testMetricTrigger_shard2_replica_n2/data/]
[junit4] 2> 292221 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 292221 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore [[testMetricTrigger_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node4/testMetricTrigger_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_7624FAE60B613F35-001/tempDir-001/node4/./testMetricTrigger_shard1_replica_n1/data/]
[junit4] 2> 292261 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 292261 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 292261 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 292261 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 292262 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 292262 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 292262 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 292262 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 292263 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@60f8320f[testMetricTrigger_shard1_replica_n1] main]
[junit4] 2> 292263 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@15540b29[testMetricTrigger_shard2_replica_n2] main]
[junit4] 2> 292264 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 292264 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 292265 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 292265 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 292265 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 292266 INFO (searcherExecutor-1469-thread-1-processing-n:127.0.0.1:59138_solr x:testMetricTrigger_shard1_replica_n1 s:shard1 c:testMetricTrigger r:core_node3) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SolrCore [testMetricTrigger_shard1_replica_n1] Registered new searcher Searcher@60f8320f[testMetricTrigger_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 292266 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 292266 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592018806009495552
[junit4] 2> 292267 INFO (searcherExecutor-1468-thread-1-processing-n:127.0.0.1:40536_solr x:testMetricTrigger_shard2_replica_n2 s:shard2 c:testMetricTrigger r:core_node4) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SolrCore [testMetricTrigger_shard2_replica_n2] Registered new searcher Searcher@15540b29[testMetricTrigger_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 292267 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592018806010544128
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40536/solr/testMetricTrigger_shard2_replica_n2/
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40536/solr/testMetricTrigger_shard2_replica_n2/ has no replicas
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59138/solr/testMetricTrigger_shard1_replica_n1/
[junit4] 2> 292274 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:59138/solr/testMetricTrigger_shard1_replica_n1/ has no replicas
[junit4] 2> 292274 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 292277 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40536/solr/testMetricTrigger_shard2_replica_n2/ shard2
[junit4] 2> 292278 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59138/solr/testMetricTrigger_shard1_replica_n1/ shard1
[junit4] 2> 292379 INFO (zkCallback-725-thread-2-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 292380 INFO (zkCallback-725-thread-1-processing-n:127.0.0.1:40536_solr) [n:127.0.0.1:40536_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 292380 INFO (zkCallback-758-thread-1-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 292380 INFO (zkCallback-758-thread-2-processing-n:127.0.0.1:59138_solr) [n:127.0.0.1:59138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testMetricTrigger/state.json] for collection [testMetricTrigger] has occurred - updating... (live nodes size: [3])
[junit4] 2> 292428 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 292429 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 292431 INFO (qtp620814082-3203) [n:127.0.0.1:59138_solr c:testMetricTrigger s:shard1 r:core_node3 x:testMetricTrigger_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testMetricTrigger_shard1_replica_n1&action=CREATE&numShards=2&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1243
[junit4] 2> 292431 INFO (qtp1419331519-3107) [n:127.0.0.1:40536_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2]
[...truncated too long message...]
etrics.MetricRegistry@24cf3e14
[junit4] 2> 2614912 INFO (coreCloseExecutor-10946-thread-2) [n:127.0.0.1:44785_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=780182118
[junit4] 2> 2614918 INFO (jetty-closer-8394-thread-3) [ ] o.a.s.c.Overseer Overseer (id=73271429979963399-127.0.0.1:45198_solr-n_0000000000) closing
[junit4] 2> 2614918 INFO (OverseerStateUpdate-73271429979963399-127.0.0.1:45198_solr-n_0000000000) [n:127.0.0.1:45198_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45198_solr
[junit4] 2> 2614918 WARN (OverseerAutoScalingTriggerThread-73271429979963399-127.0.0.1:45198_solr-n_0000000000) [n:127.0.0.1:45198_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2614920 DEBUG (jetty-closer-8394-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2614920 INFO (jetty-closer-8394-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4c33eafc{/solr,null,UNAVAILABLE}
[junit4] 2> 2614921 INFO (jetty-closer-8394-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2614921 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044ffa34240007, likely client has closed socket
[junit4] 2> 2614922 INFO (jetty-closer-8394-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6efc057b{/solr,null,UNAVAILABLE}
[junit4] 2> 2614922 INFO (jetty-closer-8394-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2616422 INFO (jetty-closer-8394-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5564f516{/solr,null,UNAVAILABLE}
[junit4] 2> 2616422 INFO (jetty-closer-8394-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2616423 ERROR (SUITE-AutoscalingHistoryHandlerTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2616423 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[7624FAE60B613F35]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44215 44215
[junit4] 2> 2616523 INFO (Thread-6172) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44215 44215
[junit4] 2> 2616524 WARN (Thread-6172) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 9 /solr/collections/AutoscalingHistoryHandlerTest_collection/terms/shard1
[junit4] 2> 8 /solr/collections/.system/terms/shard1
[junit4] 2> 4 /solr/security.json
[junit4] 2> 3 /solr/configs/.system
[junit4] 2> 3 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 3 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED
[junit4] 2> 3 /solr/configs/.system/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 22 /solr/collections/.system/state.json
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/.system/leader_elect/shard1/election/73271429979963398-core_node3-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/live_nodes
[junit4] 2> 11 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_7624FAE60B613F35-001
[junit4] 2> Feb 10, 2018 1:36:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=ko, timezone=Asia/Atyrau
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=151530040,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestLegacyField, CurrencyFieldTypeTest, ConcurrentDeleteAndCreateCollectionTest, TestDynamicFieldCollectionResource, SampleTest, PKIAuthenticationIntegrationTest, SolrMetricManagerTest, DeleteNodeTest, RulesTest, ConfigSetsAPITest, TestRawTransformer, SolrInfoBeanTest, TestClassicSimilarityFactory, TestRuleBasedAuthorizationPlugin, DistribJoinFromCollectionTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, SystemInfoHandlerTest, TestSimpleTrackingShardHandler, PrimitiveFieldTypeTest, PropertiesRequestHandlerTest, SearchRateTriggerTest, TestSQLHandlerNonCloud, DistributedDebugComponentTest, HighlighterTest, DistributedFacetPivotLargeTest, TestSweetSpotSimilarityFactory, AddReplicaTest, SaslZkACLProviderTest, TestFieldSortValues, SolrJmxReporterTest, FileUtilsTest, TestXIncludeConfig, TestFunctionQuery, TestSolrFieldCacheBean, LargeFieldTest, SchemaApiFailureTest, TestConfigSetsAPIExclusivity, DistributedFacetPivotSmallTest, TriLevelCompositeIdRoutingTest, TestSystemCollAutoCreate, SolrGangliaReporterTest, TestUseDocValuesAsStored, TestSegmentSorting, TriggerIntegrationTest, TestChildDocTransformer, ZkShardTermsTest, TestCollationFieldDocValues, TestFileDictionaryLookup, TestFastLRUCache, BlockJoinFacetSimpleTest, DirectoryFactoryTest, TestSolrJ, TestNamedUpdateProcessors, TestJsonRequest, HdfsTlogReplayBufferedWhileIndexingTest, SolrCloudExampleTest, SortByFunctionTest, TestSchemaNameResource, TestReloadDeadlock, UpdateRequestProcessorFactoryTest, TestTolerantUpdateProcessorRandomCloud, TestLRUStatsCache, TestMaxScoreQueryParser, TestStressLiveNodes, TestPayloadScoreQParserPlugin, QueryResultKeyTest, TestOnReconnectListenerSupport, TestBM25SimilarityFactory, TestBulkSchemaConcurrent, TestPivotHelperCode, AutoAddReplicasPlanActionTest, TestStressVersions, TestLeaderInitiatedRecoveryThread, PeerSyncReplicationTest, HdfsBasicDistributedZkTest, IndexSchemaTest, SpatialHeatmapFacetsTest, OverriddenZkACLAndCredentialsProvidersTest, QueryParsingTest, BaseCdcrDistributedZkTest, TestJettySolrRunner, TestSubQueryTransformerDistrib, DeleteStatusTest, FieldMutatingUpdateProcessorTest, TestDistribStateManager, SmileWriterTest, MultiThreadedOCPTest, DOMUtilTest, AnalyticsQueryTest, HttpSolrCallGetCoreTest, TestDeleteCollectionOnDownNodes, InfixSuggestersTest, RestartWhileUpdatingTest, LoggingHandlerTest, TestDistributedGrouping, ExecutePlanActionTest, TestQueryUtils, PreAnalyzedUpdateProcessorTest, TestSchemaVersionResource, CreateRoutedAliasTest, CdcrVersionReplicationTest, RequiredFieldsTest, DistributedQueryComponentOptimizationTest, TestNRTOpen, TestPerFieldSimilarity, SortSpecParsingTest, TestCharFilters, TestCloudSearcherWarming, AtomicUpdateProcessorFactoryTest, TestCloudDeleteByQuery, SoftAutoCommitTest, JsonLoaderTest, TestSchemaManager, CollectionStateFormat2Test, CdcrUpdateLogTest, DirectSolrConnectionTest, TestPartialUpdateDeduplication, MoveReplicaHDFSFailoverTest, TestLegacyTerms, TestSolrCLIRunExample, TestNoOpRegenerator, TestSolrIndexConfig, TestReplicationHandler, TestComplexPhraseLeadingWildcard, TestFieldCache, TestCoreBackup, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, OverseerTest, BasicZkTest, ClusterStateUpdateTest, TestZkChroot, TestRandomDVFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestReload, TestRangeQuery, PeerSyncTest, TestFoldingMultitermQuery, DocValuesTest, DefaultValueUpdateProcessorTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestWriterPerf, TestQueryTypes, TermsComponentTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestCollationField, MBeansHandlerTest, SOLR749Test, TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, BadComponentTest, TestStressRecovery, ClusterStateTest, TestLRUCache, UUIDFieldTest, DistributedMLTComponentTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestTolerantSearch, TestLuceneIndexBackCompat, ChaosMonkeyNothingIsSafeWithPullReplicasTest, HttpPartitionOnCommitTest, LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, MoveReplicaTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestExclusionRuleCollectionAccess, TestLockTree, TestPrepRecovery, TestRandomFlRTGCloud, TestSkipOverseerOperations, TestSolrCloudWithDelegationTokens, TestSolrCloudWithSecureImpersonation, TestStressInPlaceUpdates, ZkFailoverTest, TestCollectionsAPIViaSolrCloudCluster, TestReplicaProperties, TestRequestStatusCollectionAPI, AutoScalingHandlerTest, ComputePlanActionTest, HttpTriggerListenerTest, MetricTriggerTest, NodeAddedTriggerTest, SystemLogListenerTest, TestNodeLostTrigger, TestPolicyCloud, TestSimDistributedQueue, CdcrBidirectionalTest, CdcrBootstrapTest, HdfsRestartWhileUpdatingTest, HdfsThreadLeakTest, HdfsUnloadDistributedZkTest, HdfsWriteToMultipleCollectionsTest, StressHdfsTest, TestImplicitCoreProperties, TestReloadAndDeleteDocs, TestSolrCloudSnapshots, V2ApiIntegrationTest, V2StandaloneTest, AutoscalingHistoryHandlerTest]
[junit4] Completed [640/774 (2!)] on J1 in 15.63s, 1 test, 1 failure <<< FAILURES!
[...truncated 48614 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 363 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/363/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([5FDAC3FCD3733791:D7E74A83E9B3D63C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState(TriggerIntegrationTest.java:426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14024 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 1989490 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/init-core-data-001
[junit4] 2> 1989490 WARN (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=24 numCloses=24
[junit4] 2> 1989490 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1989492 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1989492 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001
[junit4] 2> 1989492 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1989493 INFO (Thread-6158) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1989493 INFO (Thread-6158) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1989496 ERROR (Thread-6158) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1989593 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:54329
[junit4] 2> 1989596 INFO (zkConnectionManagerCallback-7225-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989601 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044f4ebca60000, likely client has closed socket
[junit4] 2> 1989603 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1989603 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1989605 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1989605 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1989605 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1989605 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1989606 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1989606 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1989606 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d8b2b6e{/solr,null,AVAILABLE}
[junit4] 2> 1989606 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@666936c8{/solr,null,AVAILABLE}
[junit4] 2> 1989607 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1511aed{HTTP/1.1,[http/1.1]}{127.0.0.1:34705}
[junit4] 2> 1989607 INFO (jetty-launcher-7222-thread-1) [ ] o.e.j.s.Server Started @1991438ms
[junit4] 2> 1989607 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34705}
[junit4] 2> 1989608 ERROR (jetty-launcher-7222-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1989608 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@36e061ee{HTTP/1.1,[http/1.1]}{127.0.0.1:44300}
[junit4] 2> 1989608 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1989609 INFO (jetty-launcher-7222-thread-2) [ ] o.e.j.s.Server Started @1991439ms
[junit4] 2> 1989609 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1989609 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44300}
[junit4] 2> 1989609 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1989609 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:38.802Z
[junit4] 2> 1989609 ERROR (jetty-launcher-7222-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1989610 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1989610 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1989610 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1989610 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:38.803Z
[junit4] 2> 1989611 INFO (zkConnectionManagerCallback-7227-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989612 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1989612 INFO (zkConnectionManagerCallback-7229-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989613 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1989623 INFO (jetty-launcher-7222-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 1989624 INFO (jetty-launcher-7222-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 1989624 INFO (zkConnectionManagerCallback-7233-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989625 INFO (zkConnectionManagerCallback-7237-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989628 INFO (zkConnectionManagerCallback-7239-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989628 INFO (zkConnectionManagerCallback-7241-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989711 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1989711 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34705_solr
[junit4] 2> 1989712 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.Overseer Overseer (id=73270693535809542-127.0.0.1:34705_solr-n_0000000000) starting
[junit4] 2> 1989718 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34705_solr
[junit4] 2> 1989719 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1989724 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1989724 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1989725 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:34705_solr]
[junit4] 2> 1989725 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1989725 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1989726 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1989727 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1989727 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1989730 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1989733 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1989734 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44300_solr
[junit4] 2> 1989735 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1989735 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1989820 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34705.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989845 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34705.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989845 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34705.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989846 INFO (jetty-launcher-7222-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node1/.
[junit4] 2> 1989848 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44300.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989855 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44300.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989855 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44300.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1989856 INFO (jetty-launcher-7222-thread-2) [n:127.0.0.1:44300_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node2/.
[junit4] 2> 1989884 INFO (zkConnectionManagerCallback-7247-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989887 INFO (zkConnectionManagerCallback-7251-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989888 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1989888 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54329/solr ready
[junit4] 2> 1989915 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 1989915 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 1989916 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1989916 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1989917 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1989917 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1989918 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1989924 INFO (qtp870656928-27141) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1989925 INFO (qtp386038279-27138) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1989925 DEBUG (qtp870656928-27134) [n:127.0.0.1:34705_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1989927 INFO (qtp870656928-27134) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 1989927 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1989932 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:34705_solr]
[junit4] 2> 1989932 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1989932 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1989933 INFO (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1989934 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1989934 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1989934 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1989934 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1989934 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1989935 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1989935 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1989936 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1989936 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@623fd472{/solr,null,AVAILABLE}
[junit4] 2> 1989937 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@ec602c{HTTP/1.1,[http/1.1]}{127.0.0.1:39852}
[junit4] 2> 1989937 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server Started @1991768ms
[junit4] 2> 1989937 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39852}
[junit4] 2> 1989937 ERROR (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1989937 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1989937 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1989938 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1989938 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:39.131Z
[junit4] 2> 1989939 INFO (zkConnectionManagerCallback-7253-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989940 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1989949 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 1989950 INFO (zkConnectionManagerCallback-7257-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989952 INFO (zkConnectionManagerCallback-7259-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1989956 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1989958 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1989960 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39852_solr
[junit4] 2> 1989961 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1989961 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1989961 INFO (zkCallback-7258-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1989961 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1990028 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39852.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1990038 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39852.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1990038 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39852.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1990040 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:39852_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node3/.
[junit4] 2> 1990936 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1990936 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1990936 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:39852_solr at time 18510430792227744
[junit4] 2> 1991937 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1991937 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1992937 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1992937 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1993938 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1993938 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1993938 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:39852_solr] added at times [18510430792227744], now=18510433794295675
[junit4] 2> 1993938 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41c32384029ba0T3x1soud8hai70iqvybkylqxs1",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18510430792227744,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510430792227744],
[junit4] 2> "nodeNames":["127.0.0.1:39852_solr"]}}
[junit4] 2> 1993939 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
[junit4] 2> 1993940 INFO (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1993940 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41c32384029ba0T3x1soud8hai70iqvybkylqxs1",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18510430792227744,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510430792227744],
[junit4] 2> "_enqueue_time_":18510433795331082,
[junit4] 2> "nodeNames":["127.0.0.1:39852_solr"]}}
[junit4] 2> 1993941 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
[junit4] 2> 1993941 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=41c32384029ba0T3x1soud8hai70iqvybkylqxs1
[junit4] 2> 1993945 INFO (qtp870656928-27143) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1993947 INFO (qtp1582300571-27197) [n:127.0.0.1:39852_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1993949 INFO (qtp386038279-27142) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1993949 DEBUG (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1993949 INFO (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 1993950 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1993950 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:39852_solr, 127.0.0.1:34705_solr]
[junit4] 2> 1993950 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1993950 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1993951 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1993951 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1995950 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 1995988 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 1995989 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 1995989 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1995989 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1995991 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1995991 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1995991 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1995992 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1995993 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1995993 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1995993 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1995994 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@db7676d{/solr,null,AVAILABLE}
[junit4] 2> 1995994 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7bf866e9{HTTP/1.1,[http/1.1]}{127.0.0.1:37275}
[junit4] 2> 1995994 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server Started @1997825ms
[junit4] 2> 1995994 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37275}
[junit4] 2> 1995995 ERROR (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1995995 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1995995 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1995995 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1995995 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:45.188Z
[junit4] 2> 1995997 INFO (zkConnectionManagerCallback-7263-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1995997 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1996006 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 1996008 INFO (zkConnectionManagerCallback-7267-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1996010 INFO (zkConnectionManagerCallback-7269-thread-1-processing-n:127.0.0.1:37275_solr) [n:127.0.0.1:37275_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1996018 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1996020 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1996023 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37275_solr
[junit4] 2> 1996024 INFO (zkCallback-7258-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1996024 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1996024 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1996024 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1996028 INFO (zkCallback-7268-thread-1-processing-n:127.0.0.1:37275_solr) [n:127.0.0.1:37275_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1996113 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37275.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1996123 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37275.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1996123 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37275.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 1996124 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:37275_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node4/.
[junit4] 2> 1996189 INFO (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1996190 INFO (qtp1637758569-27225) [n:127.0.0.1:37275_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1996192 INFO (qtp1582300571-27202) [n:127.0.0.1:39852_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1996193 INFO (qtp386038279-27147) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1996193 DEBUG (qtp386038279-27148) [n:127.0.0.1:44300_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1996193 INFO (qtp386038279-27148) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 1996193 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1996194 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:39852_solr, 127.0.0.1:37275_solr, 127.0.0.1:34705_solr]
[junit4] 2> 1996194 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7bf866e9{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1996194 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1996194 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1698833564
[junit4] 2> 1996195 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1996195 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3522c719: rootName = solr_37275, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@763eb224
[junit4] 2> 1996195 INFO (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1996196 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1996196 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1996196 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 4
[junit4] 2> 1996201 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1996202 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1255f1a0: rootName = solr_37275, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f48e6fe
[junit4] 2> 1996207 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1996207 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@22408406: rootName = solr_37275, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@4df87cdb
[junit4] 2> 1996208 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37275_solr
[junit4] 2> 1996208 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1996208 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1996209 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1996209 INFO (zkCallback-7268-thread-1-processing-n:127.0.0.1:37275_solr) [n:127.0.0.1:37275_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1996208 INFO (zkCallback-7258-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1996208 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1996209 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1997197 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 1997197 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:37275_solr
[junit4] 2> 1997710 WARN (zkCallback-7268-thread-1-processing-n:127.0.0.1:37275_solr) [n:127.0.0.1:37275_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:37275_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:37275_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:882)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1997711 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@db7676d{/solr,null,UNAVAILABLE}
[junit4] 2> 1997711 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1998198 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 1999198 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 1999216 INFO (qtp870656928-27145) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999218 INFO (qtp1582300571-27200) [n:127.0.0.1:39852_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999219 INFO (qtp386038279-27138) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999220 DEBUG (qtp870656928-27141) [n:127.0.0.1:34705_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1999220 INFO (qtp870656928-27141) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 1999220 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1999220 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:39852_solr, 127.0.0.1:34705_solr]
[junit4] 2> 1999221 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1999221 INFO (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1999222 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:37275_solr
[junit4] 2> 1999222 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1999222 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 1999222 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:37275_solr]
[junit4] 2> 1999222 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1999222 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41c32571e3d6f7T3x1soud8hai70iqvybkylqxs3",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18510439078156023,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510439078156023],
[junit4] 2> "nodeNames":["127.0.0.1:37275_solr"]}}
[junit4] 2> 1999222 DEBUG (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_restore_trigger]
[junit4] 2> 1999223 INFO (ScheduledTrigger-8845-thread-2) [n:127.0.0.1:34705_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1999223 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41c32571e3d6f7T3x1soud8hai70iqvybkylqxs3",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18510439078156023,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510439078156023],
[junit4] 2> "_enqueue_time_":18510439078959634,
[junit4] 2> "nodeNames":["127.0.0.1:37275_solr"]}}
[junit4] 2> 1999223 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 1999224 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 5000ms
[junit4] 2> 1999224 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=41c32571e3d6f7T3x1soud8hai70iqvybkylqxs3
[junit4] 2> 1999243 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 1999243 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 8
[junit4] 2> 1999243 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1999243 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1999245 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1999245 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1999245 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1999245 INFO (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1999249 INFO (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 1999250 INFO (qtp870656928-27143) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999250 INFO (qtp1582300571-27202) [n:127.0.0.1:39852_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999251 INFO (qtp386038279-27148) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1999251 DEBUG (qtp870656928-27141) [n:127.0.0.1:34705_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1999251 INFO (qtp870656928-27141) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 1999251 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1999252 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:39852_solr, 127.0.0.1:34705_solr]
[junit4] 2> 1999252 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1999253 INFO (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1999253 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1999253 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1999253 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@ec602c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1999253 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=614049576
[junit4] 2> 1999254 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 1999254 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1999254 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1eb72d3: rootName = solr_39852, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@62241afd
[junit4] 2> 1999260 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1999260 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fc9f559: rootName = solr_39852, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3f48e6fe
[junit4] 2> 1999264 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1999264 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@237c3dca: rootName = solr_39852, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@4df87cdb
[junit4] 2> 1999265 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39852_solr
[junit4] 2> 1999266 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1999266 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1999266 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1999266 INFO (zkCallback-7258-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1999266 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1999267 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2000254 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 2000254 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:39852_solr
[junit4] 2> 2000767 WARN (zkCallback-7258-thread-1-processing-n:127.0.0.1:39852_solr) [n:127.0.0.1:39852_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:39852_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:39852_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:882)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2000768 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@623fd472{/solr,null,UNAVAILABLE}
[junit4] 2> 2000768 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2001248 INFO (OverseerCollectionConfigSetProcessor-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2001255 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 2002256 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 2002256 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:39852_solr]
[junit4] 2> 2002256 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41c325af7201beT3x1soud8hai70iqvybkylqxs5",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":18510440110883262,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510440110883262],
[junit4] 2> "nodeNames":["127.0.0.1:39852_solr"]}}
[junit4] 2> 2002256 DEBUG (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger]
[junit4] 2> 2002256 INFO (ScheduledTrigger-8845-thread-1) [n:127.0.0.1:34705_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2002257 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41c325af7201beT3x1soud8hai70iqvybkylqxs5",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":18510440110883262,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18510440110883262],
[junit4] 2> "_enqueue_time_":18510442112481998,
[junit4] 2> "nodeNames":["127.0.0.1:39852_solr"]}}
[junit4] 2> 2002258 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 5000ms
[junit4] 2> 2002258 DEBUG (AutoscalingActionExecutor-8846-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=41c325af7201beT3x1soud8hai70iqvybkylqxs5
[junit4] 2> 2002260 INFO (qtp870656928-27145) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2002262 INFO (qtp386038279-27138) [n:127.0.0.1:44300_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2002262 DEBUG (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2002263 INFO (qtp870656928-27137) [n:127.0.0.1:34705_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 2002263 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 2002263 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44300_solr, 127.0.0.1:34705_solr]
[junit4] 2> 2002263 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 2002264 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2002264 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 2004263 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
[junit4] 2> 2004290 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 2004291 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 11
[junit4] 2> 2004291 DEBUG (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 2004291 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 2004293 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2004294 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2004294 DEBUG (OverseerAutoScalingTriggerThread-73270693535809542-127.0.0.1:34705_solr-n_0000000000) [n:127.0.0.1:34705_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 2004295 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2004300 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2004300 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2004300 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2004301 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45502b87{/solr,null,AVAILABLE}
[junit4] 2> 2004301 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e57ce66{HTTP/1.1,[http/1.1]}{127.0.0.1:57537}
[junit4] 2> 2004301 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server Started @2006132ms
[junit4] 2> 2004301 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57537}
[junit4] 2> 2004302 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2004302 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2004302 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2004302 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2004302 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:53.495Z
[junit4] 2> 2004304 INFO (zkConnectionManagerCallback-7273-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004305 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2004314 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 2004316 INFO (zkConnectionManagerCallback-7277-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004319 INFO (zkConnectionManagerCallback-7279-thread-1-processing-n:127.0.0.1:57537_solr) [n:127.0.0.1:57537_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004328 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2004331 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2004334 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57537_solr
[junit4] 2> 2004336 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2004337 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2004338 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2004345 INFO (zkCallback-7278-thread-1-processing-n:127.0.0.1:57537_solr) [n:127.0.0.1:57537_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2004447 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57537.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004457 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57537.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004458 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57537.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004459 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:57537_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node5/.
[junit4] 2> 2004497 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2004498 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2004498 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2004498 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2004498 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35a74087{/solr,null,AVAILABLE}
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3604657{HTTP/1.1,[http/1.1]}{127.0.0.1:49308}
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server Started @2006330ms
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49308}
[junit4] 2> 2004499 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2004499 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:53.692Z
[junit4] 2> 2004501 INFO (zkConnectionManagerCallback-7283-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004502 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2004511 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 2004512 INFO (zkConnectionManagerCallback-7287-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004515 INFO (zkConnectionManagerCallback-7289-thread-1-processing-n:127.0.0.1:49308_solr) [n:127.0.0.1:49308_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004520 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2004522 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2004524 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49308_solr
[junit4] 2> 2004524 INFO (zkCallback-7238-thread-1-processing-n:127.0.0.1:44300_solr) [n:127.0.0.1:44300_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2004525 INFO (zkCallback-7240-thread-1-processing-n:127.0.0.1:34705_solr) [n:127.0.0.1:34705_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2004525 INFO (zkCallback-7250-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2004528 INFO (zkCallback-7288-thread-1-processing-n:127.0.0.1:49308_solr) [n:127.0.0.1:49308_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2004528 INFO (zkCallback-7278-thread-1-processing-n:127.0.0.1:57537_solr) [n:127.0.0.1:57537_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2004594 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49308.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004604 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49308.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004604 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49308.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@348c658b
[junit4] 2> 2004606 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [n:127.0.0.1:49308_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001/tempDir-001/node6/.
[junit4] 2> 2004650 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2004656 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2004656 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2004656 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2004657 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b7c5a77{/solr,null,AVAILABLE}
[junit4] 2> 2004658 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@50654140{HTTP/1.1,[http/1.1]}{127.0.0.1:34896}
[junit4] 2> 2004658 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.e.j.s.Server Started @2006489ms
[junit4] 2> 2004658 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34896}
[junit4] 2> 2004658 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2004658 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2004659 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2004659 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2004659 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T10:28:53.852Z
[junit4] 2> 2004661 INFO (zkConnectionManagerCallback-7293-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004661 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2004669 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5FDAC3FCD3733791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 2004670 INFO (zkConnectionManagerCallback-7297-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2004672 INFO (zkConnectionManagerCallback-7299-thread-1-processing-n:127.0.0.1:34896_solr) [n:127.0.0.1:34896_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[juni
[...truncated too long message...]
junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:882)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2199944 WARN (zkCallback-7403-thread-3-processing-n:127.0.0.1:54293_solr) [n:127.0.0.1:54293_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2199944 INFO (jetty-closer-7223-thread-9) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5851fb60{/solr,null,UNAVAILABLE}
[junit4] 2> 2199944 INFO (jetty-closer-7223-thread-9) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2199945 ERROR (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2199946 INFO (SUITE-TriggerIntegrationTest-seed#[5FDAC3FCD3733791]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54329 54329
[junit4] 2> 2200040 INFO (Thread-6158) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54329 54329
[junit4] 2> 2200041 WARN (Thread-6158) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 30 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 9 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 9 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 89 /solr/collections/collection1/state.json
[junit4] 2> 34 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/73270693535809574-127.0.0.1:41263_solr-n_0000000011
[junit4] 2> 3 /solr/overseer_elect/election/73270693535809571-127.0.0.1:51978_solr-n_0000000010
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/73270693535809559-core_node3-n_0000000001
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73270693535809556-core_node8-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73270693535809577-127.0.0.1:38209_solr-n_0000000012
[junit4] 2> 2 /solr/overseer_elect/election/73270693535809592-127.0.0.1:46171_solr-n_0000000017
[junit4] 2> 2 /solr/overseer_elect/election/73270693535809541-127.0.0.1:44300_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5FDAC3FCD3733791-001
[junit4] 2> Feb 10, 2018 10:32:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=pt-PT, timezone=Europe/Brussels
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=62112352,total=510132224
[junit4] 2> NOTE: All tests run in this JVM: [BadComponentTest, TestPivotHelperCode, SystemInfoHandlerTest, TestXIncludeConfig, ScriptEngineTest, DistributedQueryComponentOptimizationTest, SOLR749Test, HttpPartitionTest, LeaderFailoverAfterPartitionTest, TestFieldCacheSortRandom, TestSegmentSorting, TestCorePropertiesReload, TestSystemIdResolver, UninvertDocValuesMergePolicyTest, SolrGangliaReporterTest, HighlighterMaxOffsetTest, IndexSchemaTest, TestDistribStateManager, TestCloudDeleteByQuery, CSVRequestHandlerTest, CollectionStateFormat2Test, DistributedFacetPivotSmallTest, DirectSolrConnectionTest, TestManagedStopFilterFactory, TestBulkSchemaConcurrent, PreAnalyzedUpdateProcessorTest, TestFastLRUCache, TestFunctionQuery, DistributedMLTComponentTest, RulesTest, LoggingHandlerTest, TestLegacyFieldCache, PeerSyncTest, AutoAddReplicasPlanActionTest, ExecutePlanActionTest, TestTrieFacet, QueryEqualityTest, TestSmileRequest, TestFieldCache, TestSolrIndexConfig, QueryResultKeyTest, ChaosMonkeySafeLeaderTest, TestNRTOpen, TestPerFieldSimilarityWithDefaultOverride, PeerSyncReplicationTest, DistributedIntervalFacetingTest, TestReplicaProperties, LargeFieldTest, RestartWhileUpdatingTest, TestCursorMarkWithoutUniqueKey, TestClassicSimilarityFactory, HLLSerializationTest, PrimitiveFieldTypeTest, CreateRoutedAliasTest, AsyncCallRequestStatusResponseTest, TestSolrJ, PathHierarchyTokenizerFactoryTest, DeleteNodeTest, SimplePostToolTest, TestPullReplicaErrorHandling, QueryParsingTest, TestReplicationHandler, TestReload, TestNamedUpdateProcessors, SolrIndexMetricsTest, TestCollectionsAPIViaSolrCloudCluster, TestRawTransformer, CdcrVersionReplicationTest, TestHdfsUpdateLog, TestElisionMultitermQuery, TestImplicitCoreProperties, TestConfigSetsAPIExclusivity, TestComplexPhraseLeadingWildcard, HdfsNNFailoverTest, TestSubQueryTransformerDistrib, TestStressRecovery, TestJsonRequest, StatsReloadRaceTest, DocValuesNotIndexedTest, SortByFunctionTest, DirectUpdateHandlerOptimizeTest, TestCloudPivotFacet, UpdateLogTest, TestSurroundQueryParser, TestNumericRangeQuery64, SolrCloudExampleTest, MoveReplicaHDFSFailoverTest, TestRTGBase, TestReloadDeadlock, OverriddenZkACLAndCredentialsProvidersTest, LegacyCloudClusterPropTest, SmileWriterTest, AutoscalingHistoryHandlerTest, MultiThreadedOCPTest, TestRebalanceLeaders, TestManagedSchemaAPI, TestLeaderInitiatedRecoveryThread, TestLegacyTerms, PolyFieldTest, SpatialHeatmapFacetsTest, SolrInfoBeanTest, UpdateRequestProcessorFactoryTest, ConfigSetsAPITest, MetricUtilsTest, TestSolrCloudWithSecureImpersonation, TestPartialUpdateDeduplication, HttpSolrCallGetCoreTest, SuggesterFSTTest, OverseerTest, ZkStateWriterTest, TestRestManager, ZkControllerTest, HdfsRestartWhileUpdatingTest, FileUtilsTest, TestJavabinTupleStreamParser, SyncSliceTest, TestQueryUtils, CdcrReplicationDistributedZkTest, TestBM25SimilarityFactory, SearchHandlerTest, SaslZkACLProviderTest, TestConfigsApi, TestCoreBackup, TestStressLiveNodes, TestSkipOverseerOperations, ClassificationUpdateProcessorIntegrationTest, TestZkChroot, TestDistributedSearch, DistributedTermsComponentTest, StatsComponentTest, TestIndexSearcher, ShowFileRequestHandlerTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestFoldingMultitermQuery, SuggesterTest, TestTrie, SuggesterWFSTTest, TestUpdate, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, CacheHeaderTest, TestReversedWildcardFilterFactory, TermsComponentTest, TestValueSourceCache, TestIndexingPerformance, RequiredFieldsTest, FastVectorHighlighterTest, RegexBoostProcessorTest, TestJmxIntegration, MBeansHandlerTest, TestSimpleTrackingShardHandler, AddReplicaTest, AssignBackwardCompatibilityTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CollectionsAPISolrJTest, MoveReplicaTest, ReplaceNodeTest, TestCloudInspectUtil, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestDeleteCollectionOnDownNodes, TestExclusionRuleCollectionAccess, TestOnReconnectListenerSupport, TestRandomRequestDistribution, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TestUtilizeNode, TriLevelCompositeIdRoutingTest, ZkFailoverTest, AutoAddReplicasIntegrationTest, TriggerIntegrationTest]
[junit4] Completed [607/774 (1!)] on J1 in 210.91s, 14 tests, 1 failure <<< FAILURES!
[...truncated 48787 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 362 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/362/
6 tests failed.
FAILED: org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
Error Message:
Could not find collection : legacyFalse
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : legacyFalse
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40:150146AE763CF072]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:247)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.checkMandatoryProps(LegacyCloudClusterPropTest.java:153)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.createAndTest(LegacyCloudClusterPropTest.java:90)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud(LegacyCloudClusterPropTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread: 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread:
1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40:994EAAA21DF5DD0F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots
Error Message:
Could not load collection from ZK: SolrCloudSnapshots_restored
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: SolrCloudSnapshots_restored
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40:F24A30B4A76A790F]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots(TestSolrCloudSnapshots.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/SolrCloudSnapshots_restored/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 43 more
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40:A9FA10D6687B8447]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12091 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/init-core-data-001
[junit4] 2> 0 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 356 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.e.j.u.log Logging initialized @11743ms to org.eclipse.jetty.util.log.Slf4jLog
[junit4] 2> 412 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2454 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2491 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001
[junit4] 2> 2529 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2543 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2562 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2985 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38279
[junit4] 2> 3366 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0: ZooKeeperServer not running
[junit4] 2> 3472 ERROR (Thread-1) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 5612 INFO (zkConnectionManagerCallback-4-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 6798 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 6829 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 6829 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 6915 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 6916 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 6919 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 6920 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 6920 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 6920 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 6996 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 6996 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 6996 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 7018 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@578c132f{/solr,null,AVAILABLE}
[junit4] 2> 7032 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@245298e9{/solr,null,AVAILABLE}
[junit4] 2> 7018 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45e8e8b5{/solr,null,AVAILABLE}
[junit4] 2> 7256 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@154cb930{SSL,[ssl, http/1.1]}{127.0.0.1:34103}
[junit4] 2> 7257 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server Started @18679ms
[junit4] 2> 7257 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34103}
[junit4] 2> 7257 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@57614d98{SSL,[ssl, http/1.1]}{127.0.0.1:32827}
[junit4] 2> 7272 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server Started @18695ms
[junit4] 2> 7272 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32827}
[junit4] 2> 7276 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@155ddf10{SSL,[ssl, http/1.1]}{127.0.0.1:37794}
[junit4] 2> 7276 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server Started @18699ms
[junit4] 2> 7276 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37794}
[junit4] 2> 7393 ERROR (jetty-launcher-1-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 7393 ERROR (jetty-launcher-1-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 7402 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 7393 ERROR (jetty-launcher-1-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 7403 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 7402 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 7419 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 7419 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 7419 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 7419 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 7420 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 7436 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 7757 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T07:06:36.465Z
[junit4] 2> 7757 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T07:06:36.481Z
[junit4] 2> 7757 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T07:06:36.481Z
[junit4] 2> 7871 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 7871 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 7871 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 7904 INFO (zkConnectionManagerCallback-10-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 7918 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 7918 INFO (zkConnectionManagerCallback-9-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 7919 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 7921 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 7922 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 8869 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10022cbf9bf0002, likely client has closed socket
[junit4] 2> 10956 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38279/solr
[junit4] 2> 10956 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38279/solr
[junit4] 2> 10956 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38279/solr
[junit4] 2> 11034 INFO (zkConnectionManagerCallback-20-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11046 INFO (zkConnectionManagerCallback-21-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11048 INFO (zkConnectionManagerCallback-22-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11107 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10022cbf9bf0005, likely client has closed socket
[junit4] 2> 11107 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10022cbf9bf0006, likely client has closed socket
[junit4] 2> 11108 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10022cbf9bf0004, likely client has closed socket
[junit4] 2> 11120 INFO (zkConnectionManagerCallback-24-thread-1-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11138 INFO (zkConnectionManagerCallback-28-thread-1-processing-n:127.0.0.1:34103_solr) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 11184 INFO (zkConnectionManagerCallback-26-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 14508 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 14522 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 14585 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32827_solr
[junit4] 2> 14618 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37794_solr
[junit4] 2> 14651 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer Overseer (id=72095853501677577-127.0.0.1:32827_solr-n_0000000000) starting
[junit4] 2> 14729 INFO (zkCallback-23-thread-1-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 14731 INFO (zkCallback-25-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 15026 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 15059 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 15187 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34103_solr
[junit4] 2> 15387 INFO (zkCallback-25-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 15398 INFO (zkCallback-23-thread-1-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 15480 INFO (zkCallback-27-thread-1-processing-n:127.0.0.1:34103_solr) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 15754 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 16081 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:34103_solr"} current state version: 0
[junit4] 2> 16171 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32827_solr
[junit4] 2> 16173 INFO (zkCallback-27-thread-1-processing-n:127.0.0.1:34103_solr) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 16330 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:34103_solr
[junit4] 2> 16366 INFO (zkCallback-23-thread-1-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 16366 INFO (zkCallback-25-thread-2-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 16394 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:32827_solr"} current state version: 0
[junit4] 2> 16394 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:32827_solr
[junit4] 2> 16434 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 16565 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 17219 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:34103_solr, 127.0.0.1:32827_solr, 127.0.0.1:37794_solr]
[junit4] 2> 17220 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 17220 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 17420 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 17422 DEBUG (OverseerAutoScalingTriggerThread-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 17456 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 18495 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 18638 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32827.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18727 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37794.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18645 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34103.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18763 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34103.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18763 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34103.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18920 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32827.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18920 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37794.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18921 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32827.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 18921 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37794.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a3eb11b
[junit4] 2> 19216 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:37794_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/.
[junit4] 2> 19225 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:32827_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node3/.
[junit4] 2> 19225 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:34103_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node1/.
[junit4] 2> 19596 DEBUG (ScheduledTrigger-14-thread-2) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 20612 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 21054 INFO (zkConnectionManagerCallback-36-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 21276 INFO (zkConnectionManagerCallback-40-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 21437 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 21438 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[C406B42BD2337B40]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38279/solr ready
[junit4] 2> 21616 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 22624 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 23632 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 24640 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 25652 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 26805 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 27753 INFO (qtp2126800145-42) [n:127.0.0.1:37794_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 27816 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 27838 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 27850 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 27865 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 27867 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 27886 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 27889 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 27903 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 27915 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 27934 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 27937 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 27955 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 27957 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 28007 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 28030 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 28065 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 28067 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 28082 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 28084 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 28103 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 28154 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 28156 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 28191 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 28193 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 28212 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 28214 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 28249 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 28264 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 28266 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 28332 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 28336 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 28338 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 28340 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 28363 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 28438 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 28500 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 28502 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 28517 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 28519 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 28538 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 28548 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 28550 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 28572 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 28600 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 28610 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 28612 INFO (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 28686 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 28687 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 28719 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 28824 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 28841 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 28842 INFO (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 28894 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 28894 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 28895 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 28896 INFO (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 28930 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 28931 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 28947 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 28947 INFO (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 29005 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 29005 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 29110 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 29891 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 29953 INFO (qtp2126800145-50) [n:127.0.0.1:37794_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 30072 INFO (qtp2126800145-50) [n:127.0.0.1:37794_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 30231 INFO (qtp214836630-39) [n:127.0.0.1:32827_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 30221 INFO (qtp2093161484-33) [n:127.0.0.1:34103_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 30266 INFO (qtp2093161484-33) [n:127.0.0.1:34103_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 30301 INFO (qtp214836630-39) [n:127.0.0.1:32827_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 30338 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "node_name":"127.0.0.1:37794_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 30340 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "node_name":"127.0.0.1:37794_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 30341 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:37794/solr","node_name":"127.0.0.1:37794_solr","state":"down","type":"NRT"}
[junit4] 2> 30341 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "node_name":"127.0.0.1:37794_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 30395 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "node_name":"127.0.0.1:34103_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 30395 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "node_name":"127.0.0.1:34103_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 30395 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:34103/solr","node_name":"127.0.0.1:34103_solr","state":"down","type":"NRT"}
[junit4] 2> 30396 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "node_name":"127.0.0.1:34103_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "node_name":"127.0.0.1:37794_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 30396 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "node_name":"127.0.0.1:32827_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 30396 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "node_name":"127.0.0.1:32827_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 30396 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:32827/solr","node_name":"127.0.0.1:32827_solr","state":"down","type":"NRT"}
[junit4] 2> 30396 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:34103/solr",
[junit4] 2> "node_name":"127.0.0.1:34103_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:37794/solr",
[junit4] 2> "node_name":"127.0.0.1:37794_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:32827/solr",
[junit4] 2> "node_name":"127.0.0.1:32827_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 30512 DEBUG (OverseerStateUpdate-72095853501677577-127.0.0.1:32827_solr-n_0000000000) [n:127.0.0.1:32827_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 30533 INFO (zkCallback-23-thread-1-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30543 INFO (zkCallback-25-thread-2-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30543 INFO (zkCallback-25-thread-1-processing-n:127.0.0.1:32827_solr) [n:127.0.0.1:32827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30543 INFO (zkCallback-27-thread-1-processing-n:127.0.0.1:34103_solr) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30549 INFO (zkCallback-23-thread-2-processing-n:127.0.0.1:37794_solr) [n:127.0.0.1:37794_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30549 INFO (zkCallback-27-thread-2-processing-n:127.0.0.1:34103_solr) [n:127.0.0.1:34103_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 30898 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 31841 INFO (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 31843 INFO (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 31899 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 31937 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 31937 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 31937 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 31937 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_C406B42BD2337B40-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 31950 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:32827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 31978 WARN (qtp2126800145-50) [n:127.0.0.1:37794_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandl
[...truncated too long message...]
2> Feb 10, 2018 8:15:35 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> Feb 10, 2018 8:15:55 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread:
[junit4] 2> 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 10, 2018 8:15:55 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
[junit4] 2> Feb 10, 2018 8:15:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=fi-FI, timezone=Asia/Shanghai
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=262028056,total=524812288
[junit4] 2> NOTE: All tests run in this JVM: [TestSimDistributedQueue, LeaderFailureAfterFreshStartTest, ConnectionManagerTest, ExplicitHLLTest, AssignTest, DeleteReplicaTest, HdfsLockFactoryTest, CollectionsAPIAsyncDistributedZkTest, ReplicaListTransformerTest, IndexSchemaRuntimeFieldTest, CollectionsAPISolrJTest, TestUtils, TestQuerySenderListener, ConnectionReuseTest, TestXmlQParser, TestOrdValues, TestSolrCoreSnapshots, SolrXmlInZkTest, TestSolrConfigHandler, TestNumericTerms32, DistributedSuggestComponentTest, BlobRepositoryCloudTest, TestValueSourceCache, TestRealTimeGet, GraphQueryTest, TestLazyCores, TestUnifiedSolrHighlighter, TestExecutePlanAction, TestMultiValuedNumericRangeQuery, TestAddFieldRealTimeGet, SharedFSAutoReplicaFailoverTest, SecurityConfHandlerTest, TestSolrConfigHandlerCloud, TestRandomDVFaceting, RegexBytesRefFilterTest, TestSolr4Spatial2, ShowFileRequestHandlerTest, TestFreeTextSuggestions, TimeRoutedAliasUpdateProcessorTest, SimpleCollectionCreateDeleteTest, ConvertedLegacyTest, DistributedFacetPivotLongTailTest, DataDrivenBlockJoinTest, CoreAdminHandlerTest, MetricsConfigTest, DistributedExpandComponentTest, CoreAdminCreateDiscoverTest, ClusterStateTest, LeaderElectionIntegrationTest, CurrencyRangeFacetCloudTest, TestUtilizeNode, UniqFieldsUpdateProcessorFactoryTest, TestDFRSimilarityFactory, NodeAddedTriggerTest, TestFieldCacheReopen, TestMissingGroups, HDFSCollectionsAPITest, CustomCollectionTest, LukeRequestHandlerTest, TestReloadAndDeleteDocs, DateRangeFieldTest, NotRequiredUniqueKeyTest, TestScoreJoinQPNoScore, TestApiFramework, TestDynamicFieldResource, AliasIntegrationTest, TestTlogReplica, TestAnalyzeInfixSuggestions, MoreLikeThisHandlerTest, WrapperMergePolicyFactoryTest, ReturnFieldsTest, HdfsRecoverLeaseTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestUseDocValuesAsStored2, CircularListTest, UnloadDistributedZkTest, BigEndianAscendingWordDeserializerTest, TestRestoreCore, ResponseHeaderTest, TestInitQParser, OverseerCollectionConfigSetProcessorTest, TestRequestForwarding, TestNumericTokenStream, TestSuggestSpellingConverter, ForceLeaderTest, DateMathParserTest, ResponseLogComponentTest, CacheHeaderTest, ClusterStateUpdateTest, TestIntervalFaceting, TestSolrDeletionPolicy2, TestLFUCache, DistributedSpellCheckComponentTest, CleanupOldIndexTest, TestRangeQuery, DistanceUnitsTest, ImplicitSnitchTest, TestExpandComponent, DistributedFacetPivotWhiteBoxTest, SolrTestCaseJ4Test, BlockJoinFacetDistribTest, HdfsAutoAddReplicasIntegrationTest, AutoScalingHandlerTest, TestRandomFlRTGCloud, TestEmbeddedSolrServerConstructors, ReplaceNodeTest, TestCloudPseudoReturnFields, TestManagedSynonymFilterFactory, TestCustomStream, WordBreakSolrSpellCheckerTest, TestHdfsCloudBackupRestore, TestQueryTypes, AnalysisAfterCoreReloadTest, CloudMLTQParserTest, TestConfigSetsAPIZkFailure, TestSerializedLuceneMatchVersion, HLLUtilTest, SystemLogListenerTest, LIRRollingUpdatesTest, BlockDirectoryTest, TestBadConfig, NumericFieldsTest, TestPushWriter, TestFaceting, TestRecovery, ZkControllerTest, TestStressReorder, TestReload, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, TestSort, DistributedQueryElevationComponentTest, SpellCheckCollatorTest, DocValuesTest, TestCSVResponseWriter, BinaryUpdateRequestHandlerTest, TestComponentsName, HighlighterConfigTest, TestSolrIndexConfig, CopyFieldTest, SolrIndexConfigTest, TestStressRecovery, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestTolerantSearch, TestLuceneIndexBackCompat, AsyncCallRequestStatusResponseTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteInactiveReplicaTest, DistribCursorPagingTest, MigrateRouteKeyTest, MoveReplicaHDFSTest, SSLMigrationTest, TestCloudInspectUtil, TestCloudPivotFacet, TestConfigSetsAPIExclusivity, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestLeaderInitiatedRecoveryThread]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLeaderInitiatedRecoveryThread -Dtests.seed=C406B42BD2337B40 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=fi-FI -Dtests.timezone=Asia/Shanghai -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J2 | TestLeaderInitiatedRecoveryThread (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread:
[junit4] > 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=22998, name=qtp1585610140-22998, state=TIMED_WAITING, group=TGRP-TestLeaderInitiatedRecoveryThread]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([C406B42BD2337B40]:0)
[junit4] Completed [565/774 (5!)] on J2 in 71.42s, 1 test, 2 errors <<< FAILURES!
[...truncated 48873 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 361 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/361/
2 tests failed.
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([ED6EF1506E119F1C:653ACE8AC0EDF2E4]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([ED6EF1506E119F1C:CB044F69751E3070]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger(TriggerIntegrationTest.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12583 lines...]
[junit4] Suite: org.apache.solr.cloud.ReplaceNodeNoTargetTest
[junit4] 2> 581052 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/init-core-data-001
[junit4] 2> 581061 WARN (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 581061 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 581062 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 581062 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001
[junit4] 2> 581063 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 581063 INFO (Thread-2740) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 581063 INFO (Thread-2740) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 581064 ERROR (Thread-2740) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 581163 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:57784
[junit4] 2> 581165 INFO (zkConnectionManagerCallback-1324-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581170 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581170 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581170 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581170 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581170 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581175 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 581177 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581177 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581177 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581179 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581179 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581179 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59324765{/solr,null,AVAILABLE}
[junit4] 2> 581179 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581183 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581183 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581183 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a9a5594{HTTP/1.1,[http/1.1]}{127.0.0.1:32919}
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.e.j.s.Server Started @583093ms
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32919}
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581187 ERROR (jetty-launcher-1321-thread-6) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@431542e2{/solr,null,AVAILABLE}
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@683c1353{/solr,null,AVAILABLE}
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.553Z
[junit4] 2> 581187 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49d66e27{HTTP/1.1,[http/1.1]}{127.0.0.1:53554}
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.e.j.s.Server Started @583094ms
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53554}
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@190694d9{HTTP/1.1,[http/1.1]}{127.0.0.1:58626}
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.e.j.s.Server Started @583094ms
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58626}
[junit4] 2> 581188 ERROR (jetty-launcher-1321-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581188 ERROR (jetty-launcher-1321-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.554Z
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581188 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.554Z
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f93a9c4{/solr,null,AVAILABLE}
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5730212{/solr,null,AVAILABLE}
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4f9b2582{HTTP/1.1,[http/1.1]}{127.0.0.1:55107}
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@50a0af09{HTTP/1.1,[http/1.1]}{127.0.0.1:48750}
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.e.j.s.Server Started @583095ms
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-4) [ ] o.e.j.s.Server Started @583095ms
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55107}
[junit4] 2> 581189 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48750}
[junit4] 2> 581190 ERROR (jetty-launcher-1321-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581190 ERROR (jetty-launcher-1321-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.556Z
[junit4] 2> 581190 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.556Z
[junit4] 2> 581191 INFO (zkConnectionManagerCallback-1326-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581191 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581195 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c100859{/solr,null,AVAILABLE}
[junit4] 2> 581197 INFO (zkConnectionManagerCallback-1332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3bbed96e{HTTP/1.1,[http/1.1]}{127.0.0.1:48992}
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-2) [ ] o.e.j.s.Server Started @583103ms
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48992}
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581197 ERROR (jetty-launcher-1321-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 581197 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581198 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581198 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T23:34:55.564Z
[junit4] 2> 581199 INFO (zkConnectionManagerCallback-1330-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581199 INFO (zkConnectionManagerCallback-1328-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581200 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581203 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581203 INFO (jetty-launcher-1321-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581203 INFO (zkConnectionManagerCallback-1334-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581207 INFO (zkConnectionManagerCallback-1340-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581211 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581211 INFO (zkConnectionManagerCallback-1336-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581211 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044cf83ccd0007, likely client has closed socket
[junit4] 2> 581212 INFO (jetty-launcher-1321-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581212 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 581213 INFO (zkConnectionManagerCallback-1343-thread-1-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581217 INFO (jetty-launcher-1321-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581217 INFO (jetty-launcher-1321-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581225 INFO (zkConnectionManagerCallback-1346-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581227 INFO (jetty-launcher-1321-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581227 INFO (jetty-launcher-1321-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57784/solr
[junit4] 2> 581231 INFO (zkConnectionManagerCallback-1352-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581231 INFO (zkConnectionManagerCallback-1354-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581231 INFO (zkConnectionManagerCallback-1360-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581235 INFO (zkConnectionManagerCallback-1362-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581240 INFO (zkConnectionManagerCallback-1368-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581240 INFO (zkConnectionManagerCallback-1366-thread-1-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581240 INFO (zkConnectionManagerCallback-1373-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581240 INFO (zkConnectionManagerCallback-1370-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581243 INFO (zkConnectionManagerCallback-1364-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581256 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581256 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581256 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581259 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58626_solr
[junit4] 2> 581260 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.c.Overseer Overseer (id=73268123000438802-127.0.0.1:58626_solr-n_0000000000) starting
[junit4] 2> 581260 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53554_solr
[junit4] 2> 581262 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55107_solr
[junit4] 2> 581287 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 581287 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 581292 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 581304 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58626_solr
[junit4] 2> 581305 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 581305 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 581311 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 581332 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 581334 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581336 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48750_solr
[junit4] 2> 581344 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581344 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581344 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581351 INFO (zkCallback-1365-thread-1-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581358 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 581359 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581361 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32919_solr
[junit4] 2> 581361 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 581361 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 581362 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 581375 INFO (zkCallback-1365-thread-1-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 581378 INFO (zkCallback-1342-thread-1-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 581381 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 581383 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581384 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48992_solr
[junit4] 2> 581385 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581385 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581385 INFO (zkCallback-1365-thread-1-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581385 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581403 INFO (zkCallback-1342-thread-1-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581415 INFO (zkCallback-1369-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 581421 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53554.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581453 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58626.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581468 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53554.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581468 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53554.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581472 INFO (jetty-launcher-1321-thread-3) [n:127.0.0.1:53554_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node3/.
[junit4] 2> 581477 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55107.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581489 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32919.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581504 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48992.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581508 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58626.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581508 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58626.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581511 INFO (jetty-launcher-1321-thread-5) [n:127.0.0.1:58626_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node5/.
[junit4] 2> 581516 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55107.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581516 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55107.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581517 INFO (jetty-launcher-1321-thread-1) [n:127.0.0.1:55107_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node1/.
[junit4] 2> 581519 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48750.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581524 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48992.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581524 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48992.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581525 INFO (jetty-launcher-1321-thread-2) [n:127.0.0.1:48992_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node2/.
[junit4] 2> 581527 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32919.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581527 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32919.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581528 INFO (jetty-launcher-1321-thread-6) [n:127.0.0.1:32919_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node6/.
[junit4] 2> 581532 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48750.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581532 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48750.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 581533 INFO (jetty-launcher-1321-thread-4) [n:127.0.0.1:48750_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node4/.
[junit4] 2> 581775 INFO (zkConnectionManagerCallback-1386-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581778 INFO (zkConnectionManagerCallback-1390-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581778 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 581779 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:57784/solr ready
[junit4] 2> 581815 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[ED6EF1506E119F1C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 581816 INFO (zkConnectionManagerCallback-1392-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581817 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[ED6EF1506E119F1C]) [ ] o.a.s.c.ReplaceNodeNoTargetTest total_jettys: 6
[junit4] 2> 581822 INFO (qtp506653784-7679) [n:127.0.0.1:32919_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581823 INFO (qtp658803029-7681) [n:127.0.0.1:48750_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581825 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581827 INFO (qtp1716327794-7673) [n:127.0.0.1:53554_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581829 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581830 INFO (qtp2060605151-7715) [n:127.0.0.1:58626_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581831 INFO (qtp1306231033-7665) [n:127.0.0.1:55107_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=11
[junit4] 2> 581832 INFO (qtp1306231033-7675) [n:127.0.0.1:55107_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&collection.configName=conf1&name=replacenodetest_coll_notarget&nrtReplicas=2&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 581833 INFO (OverseerThreadFactory-2099-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll_notarget
[junit4] 2> 581840 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581841 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581844 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581845 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581846 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581847 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 581856 INFO (OverseerThreadFactory-2099-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18471206885 sessionWrapper.createTime 18471206876208181, this.sessionWrapper.createTime 18471206876208181
[junit4] 2> 581964 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58626/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581966 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:55107/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581967 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53554/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581969 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48992/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581970 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48750/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581972 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:32919/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581974 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:32919/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581976 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48750/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581977 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48992/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 581979 INFO (OverseerStateUpdate-73268123000438802-127.0.0.1:58626_solr-n_0000000000) [n:127.0.0.1:58626_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53554/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 582183 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582184 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582184 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582184 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n6&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582184 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n4&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582184 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582184 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582184 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582186 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n8&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582186 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n10&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582186 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582186 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 582188 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n12&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582193 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node17&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n14&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582193 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n16&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582196 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node20&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n18&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 582297 INFO (zkCallback-1363-thread-1-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582298 INFO (zkCallback-1342-thread-1-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582298 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582298 INFO (zkCallback-1365-thread-1-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582298 INFO (zkCallback-1369-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582298 INFO (zkCallback-1372-thread-1-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582300 INFO (zkCallback-1363-thread-2-processing-n:127.0.0.1:53554_solr) [n:127.0.0.1:53554_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1372-thread-2-processing-n:127.0.0.1:58626_solr) [n:127.0.0.1:58626_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1342-thread-1-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1342-thread-2-processing-n:127.0.0.1:32919_solr) [n:127.0.0.1:32919_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1365-thread-2-processing-n:127.0.0.1:48750_solr) [n:127.0.0.1:48750_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1369-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 582301 INFO (zkCallback-1367-thread-2-processing-n:127.0.0.1:55107_solr) [n:127.0.0.1:55107_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 583211 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583212 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583221 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583224 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583225 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583229 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583230 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583231 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n2] Schema name=cloud-dynamic
[junit4] 2> 583231 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583245 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583248 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 583250 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n1] Schema name=cloud-dynamic
[junit4] 2> 583252 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n16] Schema name=cloud-dynamic
[junit4] 2> 583254 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n10] Schema name=cloud-dynamic
[junit4] 2> 583255 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n8] Schema name=cloud-dynamic
[junit4] 2> 583255 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n12] Schema name=cloud-dynamic
[junit4] 2> 583278 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n4] Schema name=cloud-dynamic
[junit4] 2> 583278 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583278 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n1' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583279 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58626.solr.core.replacenodetest_coll_notarget.shard1.replica_n1' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583279 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583279 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node5/replacenodetest_coll_notarget_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node5/./replacenodetest_coll_notarget_shard1_replica_n1/data/]
[junit4] 2> 583285 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n6] Schema name=cloud-dynamic
[junit4] 2> 583260 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n18] Schema name=cloud-dynamic
[junit4] 2> 583257 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n14] Schema name=cloud-dynamic
[junit4] 2> 583344 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583344 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n10' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583344 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583344 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n6' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583345 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32919.solr.core.replacenodetest_coll_notarget.shard3.replica_n10' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583345 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48992.solr.core.replacenodetest_coll_notarget.shard2.replica_n6' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583345 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583345 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583345 INFO (qtp506653784-7685) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n10] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node6/replacenodetest_coll_notarget_shard3_replica_n10], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node6/./replacenodetest_coll_notarget_shard3_replica_n10/data/]
[junit4] 2> 583345 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node2/replacenodetest_coll_notarget_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard2_replica_n6/data/]
[junit4] 2> 583348 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583348 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n8' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583348 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48750.solr.core.replacenodetest_coll_notarget.shard3.replica_n8' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583348 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583348 INFO (qtp658803029-7702) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n8] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node4/replacenodetest_coll_notarget_shard3_replica_n8], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard3_replica_n8/data/]
[junit4] 2> 583369 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583369 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n2' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583370 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55107.solr.core.replacenodetest_coll_notarget.shard1.replica_n2' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583370 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583370 INFO (qtp1306231033-7704) [n:127.0.0.1:55107_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node1/replacenodetest_coll_notarget_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node1/./replacenodetest_coll_notarget_shard1_replica_n2/data/]
[junit4] 2> 583373 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583373 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n18' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583373 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53554.solr.core.replacenodetest_coll_notarget.shard5.replica_n18' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n18') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583373 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583376 INFO (qtp1716327794-7710) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n18] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node3/replacenodetest_coll_notarget_shard5_replica_n18], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node3/./replacenodetest_coll_notarget_shard5_replica_n18/data/]
[junit4] 2> 583379 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583379 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n16' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583381 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583381 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n12' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583384 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583384 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n4' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583385 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53554.solr.core.replacenodetest_coll_notarget.shard2.replica_n4' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583385 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583385 INFO (qtp1716327794-7700) [n:127.0.0.1:53554_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node3/replacenodetest_coll_notarget_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node3/./replacenodetest_coll_notarget_shard2_replica_n4/data/]
[junit4] 2> 583386 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 583386 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n14' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 583386 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48750.solr.core.replacenodetest_coll_notarget.shard4.replica_n14' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583386 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583387 INFO (qtp658803029-7709) [n:127.0.0.1:48750_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n14] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node4/replacenodetest_coll_notarget_shard4_replica_n14], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard4_replica_n14/data/]
[junit4] 2> 583389 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32919.solr.core.replacenodetest_coll_notarget.shard4.replica_n12' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583389 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583389 INFO (qtp506653784-7693) [n:127.0.0.1:32919_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n12] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node6/replacenodetest_coll_notarget_shard4_replica_n12], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node6/./replacenodetest_coll_notarget_shard4_replica_n12/data/]
[junit4] 2> 583402 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48992.solr.core.replacenodetest_coll_notarget.shard5.replica_n16' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n16') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5620cdec
[junit4] 2> 583402 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 583402 INFO (qtp836621078-7674) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n16] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node2/replacenodetest_coll_notarget_shard5_replica_n16], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ReplaceNodeNoTargetTest_ED6EF1506E119F1C-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard5_replica_n16/data/]
[junit4] 2> 583479 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 583479 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 583482 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 583482 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 583501 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 583501 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 583502 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@7536ab77[replacenodetest_coll_notarget_shard2_replica_n6] main]
[junit4] 2> 583502 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 583502 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 583503 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 583503 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 583504 INFO (qtp836621078-7697) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 583507 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@631ce85f[replacenodetest_coll_notarget_shard1_replica_n1] main]
[junit4] 2> 583508 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 583508 INFO (qtp2060605151-7708) [n:127.0.0.1:58626_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 583524 INFO (searcherExecutor-2114-thread-1-processing-n:127.0.0.1:48992_solr x:replacenodetest_coll_notarget_shard2_replica_n6 s:shard2 c:replacenodetest_coll_notarget r:core_node9) [n:127.0.0.1:48992_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore [
[...truncated too long message...]
nodes from ZooKeeper... (4) -> (0)
[junit4] 2> 979635 INFO (jetty-closer-2170-thread-7) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6e25f751{/solr,null,UNAVAILABLE}
[junit4] 2> 979636 INFO (jetty-closer-2170-thread-12) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 979636 INFO (jetty-closer-2170-thread-7) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 979637 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044cfb1365003f, likely client has closed socket
[junit4] 2> 979643 INFO (coreCloseExecutor-3548-thread-1) [n:127.0.0.1:44911_solr c:testMetricTrigger s:shard1 r:core_node8 x:testMetricTrigger_shard1_replica_n7] o.a.s.c.SolrCore [testMetricTrigger_shard1_replica_n7] CLOSING SolrCore org.apache.solr.core.SolrCore@40ecf317
[junit4] 2> 979643 INFO (coreCloseExecutor-3548-thread-1) [n:127.0.0.1:44911_solr c:testMetricTrigger s:shard1 r:core_node8 x:testMetricTrigger_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testMetricTrigger.shard1.replica_n7, tag=1089270551
[junit4] 2> 979643 INFO (coreCloseExecutor-3548-thread-1) [n:127.0.0.1:44911_solr c:testMetricTrigger s:shard1 r:core_node8 x:testMetricTrigger_shard1_replica_n7] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@59f0dcbb: rootName = solr_44911, domain = solr.core.testMetricTrigger.shard1.replica_n7, service url = null, agent id = null] for registry solr.core.testMetricTrigger.shard1.replica_n7 / com.codahale.metrics.MetricRegistry@3ed4020c
[junit4] 2> 979644 INFO (coreCloseExecutor-3544-thread-1) [n:127.0.0.1:53730_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard2.leader, tag=426328513
[junit4] 2> 979647 INFO (zkCallback-2291-thread-3-processing-n:127.0.0.1:44911_solr) [n:127.0.0.1:44911_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44911_solr
[junit4] 2> 979647 INFO (jetty-closer-2170-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5663b0f8{/solr,null,UNAVAILABLE}
[junit4] 2> 979647 INFO (jetty-closer-2170-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 979648 INFO (coreCloseExecutor-3545-thread-1) [n:127.0.0.1:49286_solr c:testMetricTrigger s:shard1 r:core_node6 x:testMetricTrigger_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard1.leader, tag=1971056059
[junit4] 2> 979651 INFO (jetty-closer-2170-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@47400f3a{/solr,null,UNAVAILABLE}
[junit4] 2> 979651 INFO (jetty-closer-2170-thread-8) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 979655 INFO (coreCloseExecutor-3548-thread-1) [n:127.0.0.1:44911_solr c:testMetricTrigger s:shard1 r:core_node8 x:testMetricTrigger_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard1.leader, tag=1089270551
[junit4] 2> 979657 INFO (jetty-closer-2170-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@f4d7ba7{/solr,null,UNAVAILABLE}
[junit4] 2> 979657 INFO (jetty-closer-2170-thread-6) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 981135 INFO (jetty-closer-2170-thread-13) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@232d4fcf{/solr,null,UNAVAILABLE}
[junit4] 2> 981135 INFO (jetty-closer-2170-thread-13) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 981136 ERROR (SUITE-TriggerIntegrationTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 981137 INFO (SUITE-TriggerIntegrationTest-seed#[ED6EF1506E119F1C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43201 43201
[junit4] 2> 981302 INFO (Thread-3290) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43201 43201
[junit4] 2> 981302 WARN (Thread-3290) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 35 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 10 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 8 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 100 /solr/collections/collection1/state.json
[junit4] 2> 37 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 5 /solr/overseer_elect/election/73268135190659113-127.0.0.1:49286_solr-n_0000000012
[junit4] 2> 2 /solr/overseer_elect/election/73268135190659107-127.0.0.1:44911_solr-n_0000000010
[junit4] 2> 2 /solr/overseer_elect/election/73268135190659129-127.0.0.1:60096_solr-n_0000000017
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73268135190659089-core_node3-n_0000000000
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73268135190659077-core_node8-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73268135190659098-127.0.0.1:33114_solr-n_0000000007
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73268135190659086-core_node6-n_0000000002
[junit4] 2> 2 /solr/overseer_elect/election/73268135190659092-127.0.0.1:36096_solr-n_0000000005
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_ED6EF1506E119F1C-001
[junit4] 2> Feb 09, 2018 11:41:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), x_s=FSTOrd50}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=824, maxMBSortInHeap=6.488910509318565, sim=RandomSimilarity(queryNorm=false): {}, locale=th, timezone=Israel
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=241557512,total=505413632
[junit4] 2> NOTE: All tests run in this JVM: [SearchRateTriggerTest, DistributedQueryComponentOptimizationTest, SaslZkACLProviderTest, ZkShardTermsTest, TestJavabinTupleStreamParser, DirectoryFactoryTest, FieldMutatingUpdateProcessorTest, QueryParsingTest, AutoscalingHistoryHandlerTest, CdcrVersionReplicationTest, TestSystemIdResolver, RequiredFieldsTest, TestLargeCluster, SpellCheckCollatorWithCollapseTest, TestEmbeddedSolrServerAdminHandler, BlockJoinFacetSimpleTest, TestLegacyFieldCache, ConfigSetsAPITest, TestCloudSearcherWarming, TestMaxScoreQueryParser, TestFieldCacheSortRandom, TestSolr4Spatial, BasicZkTest, SolrGangliaReporterTest, SimplePostToolTest, SolrCloudExampleTest, LegacyCloudClusterPropTest, UninvertDocValuesMergePolicyTest, TestImpersonationWithHadoopAuth, QueryElevationComponentTest, BaseCdcrDistributedZkTest, SchemaApiFailureTest, TestCoreBackup, BadComponentTest, TestNRTOpen, HdfsNNFailoverTest, HLLSerializationTest, SOLR749Test, TestCloudPivotFacet, BitVectorTest, TestSchemaVersionResource, TestSimpleTextCodec, StressHdfsTest, SortByFunctionTest, TestFieldCache, TestRebalanceLeaders, TestCloudDeleteByQuery, DistributedFacetSimpleRefinementLongTailTest, RestartWhileUpdatingTest, CreateRoutedAliasTest, TestMaxTokenLenTokenizer, TestPostingsSolrHighlighter, DeleteStatusTest, CdcrReplicationDistributedZkTest, TestFoldingMultitermQuery, TestStressVersions, SuggesterFSTTest, TestFileDictionaryLookup, DistribJoinFromCollectionTest, AnalyticsQueryTest, TestHdfsUpdateLog, InfixSuggestersTest, RulesTest, SolrIndexMetricsTest, HdfsTlogReplayBufferedWhileIndexingTest, TestSchemaNameResource, TestRawTransformer, SignatureUpdateProcessorFactoryTest, TestLegacyTerms, OverriddenZkACLAndCredentialsProvidersTest, TestManagedSchemaAPI, ShardRoutingCustomTest, ReplaceNodeNoTargetTest, TestCorePropertiesReload, SolrShardReporterTest, TestElisionMultitermQuery, TestSolrCloudSnapshots, OverseerTest, TestTrie, TestBM25SimilarityFactory, TestBulkSchemaConcurrent, LargeFieldTest, SuggesterTest, TestFieldResource, CdcrUpdateLogTest, TestHashQParserPlugin, TestOnReconnectListenerSupport, TestNumericRangeQuery64, TestClassicSimilarityFactory, TestSubQueryTransformerDistrib, SolrCoreTest, UUIDUpdateProcessorFallbackTest, TestSimpleTrackingShardHandler, CollectionStateFormat2Test, TestExportWriter, TestDeleteCollectionOnDownNodes, HdfsRestartWhileUpdatingTest, TriggerIntegrationTest]
[junit4] Completed [224/774 (2!)] on J1 in 215.04s, 14 tests, 1 failure <<< FAILURES!
[...truncated 50070 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 360 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/360/
11 tests failed.
FAILED: org.apache.solr.cloud.LeaderFailureAfterFreshStartTest.test
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:36B4856C32DC8D69]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:117)
at org.apache.solr.cloud.LeaderFailureAfterFreshStartTest.waitTillNodesActive(LeaderFailureAfterFreshStartTest.java:209)
at org.apache.solr.cloud.LeaderFailureAfterFreshStartTest.restartNodes(LeaderFailureAfterFreshStartTest.java:174)
at org.apache.solr.cloud.LeaderFailureAfterFreshStartTest.test(LeaderFailureAfterFreshStartTest.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 47 more
FAILED: org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
Error Message:
Could not find collection : legacyFalse
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : legacyFalse
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:6FE74833382F6BA3]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:247)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.checkMandatoryProps(LegacyCloudClusterPropTest.java:153)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.createAndTest(LegacyCloudClusterPropTest.java:90)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud(LegacyCloudClusterPropTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.OverseerTaskQueueTest.testDistributedQueueBlocking
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:FB4AC8CFD8725CE5]:0)
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.apache.solr.cloud.DistributedQueueTest.testDistributedQueueBlocking(DistributedQueueTest.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestTlogReplica.testKillLeader
Error Message:
Stack Trace:
java.lang.NullPointerException
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.TestTlogReplica.doReplaceLeader(TestTlogReplica.java:427)
at org.apache.solr.cloud.TestTlogReplica.testKillLeader(TestTlogReplica.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestTlogReplica.testOutOfOrderDBQWithInPlaceUpdates
Error Message:
Error from server at https://127.0.0.1:48665/solr: Request says it is coming from leader, but we are the leader
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48665/solr: Request says it is coming from leader, but we are the leader
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:3821425BC3713671]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.cloud.TestTlogReplica.testOutOfOrderDBQWithInPlaceUpdates(TestTlogReplica.java:672)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:4EC8D39C3C836DE]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:E3A8A43F53E646DE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:E3A8A43F53E646DE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10020_solr should have been deleted
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10020_solr should have been deleted
at __randomizedtesting.SeedInfo.seed([BEE0BAB69C20E091:A65A32BA92152D7E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12361 lines...]
[junit4] Suite: org.apache.solr.cloud.LegacyCloudClusterPropTest
[junit4] 2> 1071431 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/init-core-data-001
[junit4] 2> 1071432 WARN (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1071442 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1071454 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1071454 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001
[junit4] 2> 1071454 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1072354 INFO (Thread-684) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1072354 INFO (Thread-684) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1072423 ERROR (Thread-684) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1072458 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46602
[junit4] 2> 1072498 INFO (zkConnectionManagerCallback-404-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1072542 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1044a8629310000, likely client has closed socket
[junit4] 2> 1072702 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1072702 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1072702 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1072703 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1072703 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13e56c7f{/solr,null,AVAILABLE}
[junit4] 2> 1072722 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4bff3d3d{HTTP/1.1,[http/1.1]}{127.0.0.1:36013}
[junit4] 2> 1072722 INFO (jetty-launcher-401-thread-1) [ ] o.e.j.s.Server Started @1095572ms
[junit4] 2> 1072722 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36013}
[junit4] 2> 1072723 ERROR (jetty-launcher-401-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1072723 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1072723 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1072723 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1072723 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T12:11:05.242Z
[junit4] 2> 1072798 INFO (zkConnectionManagerCallback-406-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1072798 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1072900 INFO (jetty-launcher-401-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46602/solr
[junit4] 2> 1073120 INFO (zkConnectionManagerCallback-410-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1073128 INFO (zkConnectionManagerCallback-412-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1074258 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1074258 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36013_solr
[junit4] 2> 1074259 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.c.Overseer Overseer (id=73265434021920771-127.0.0.1:36013_solr-n_0000000000) starting
[junit4] 2> 1074355 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36013_solr
[junit4] 2> 1074382 INFO (OverseerStateUpdate-73265434021920771-127.0.0.1:36013_solr-n_0000000000) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1075581 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1075745 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1075764 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1075810 INFO (jetty-launcher-401-thread-1) [n:127.0.0.1:36013_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/.
[junit4] 2> 1076455 INFO (zkConnectionManagerCallback-416-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1076534 INFO (zkConnectionManagerCallback-420-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1076556 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1076556 INFO (SUITE-LegacyCloudClusterPropTest-seed#[BEE0BAB69C20E091]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46602/solr ready
[junit4] 2> 1076595 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSwitchLegacyCloud
[junit4] 2> 1076655 INFO (qtp2023301172-2277) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1076677 INFO (qtp2023301172-2277) [n:127.0.0.1:36013_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 1076705 INFO (qtp2023301172-2279) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyTrue&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1077271 INFO (OverseerThreadFactory-742-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection legacyTrue
[junit4] 2> 1077433 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyTrue_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyTrue&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1077446 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1077548 INFO (zkCallback-411-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1078705 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1078944 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema [legacyTrue_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1078965 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1078965 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'legacyTrue_shard1_replica_n1' using configuration from collection legacyTrue, trusted=true
[junit4] 2> 1079163 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.core.legacyTrue.shard1.replica_n1' (registry 'solr.core.legacyTrue.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1079163 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1079163 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [[legacyTrue_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/legacyTrue_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/./legacyTrue_shard1_replica_n1/data/]
[junit4] 2> 1079658 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1079658 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1079659 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1079659 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1079714 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a236cc0[legacyTrue_shard1_replica_n1] main]
[junit4] 2> 1079743 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1079744 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1079744 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1079745 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591925300017496064
[junit4] 2> 1079762 INFO (searcherExecutor-745-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [legacyTrue_shard1_replica_n1] Registered new searcher Searcher@a236cc0[legacyTrue_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/ has no replicas
[junit4] 2> 1079846 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1079863 INFO (zkCallback-411-thread-2-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1079864 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/ shard1
[junit4] 2> 1079865 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1079882 INFO (qtp2023301172-2282) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyTrue_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyTrue&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2448
[junit4] 2> 1079884 INFO (qtp2023301172-2279) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1079982 INFO (zkCallback-411-thread-2-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1080904 INFO (qtp2023301172-2279) [n:127.0.0.1:36013_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyTrue&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=4198
[junit4] 2> 1080933 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@4bff3d3d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1080950 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=964961590
[junit4] 2> 1080958 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1080958 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d6300d5: rootName = solr_36013, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@77caa671
[junit4] 2> 1080980 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1080980 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2cb4b48: rootName = solr_36013, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@34de1aa0
[junit4] 2> 1080996 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1080996 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@cc19fac: rootName = solr_36013, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1cd34c9f
[junit4] 2> 1081007 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36013_solr
[junit4] 2> 1081008 INFO (zkCallback-419-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1081008 INFO (zkCallback-411-thread-2-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1081008 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1081062 INFO (coreCloseExecutor-750-thread-1) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [legacyTrue_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c490e3c
[junit4] 2> 1081063 INFO (coreCloseExecutor-750-thread-1) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.legacyTrue.shard1.replica_n1, tag=474549820
[junit4] 2> 1081063 INFO (coreCloseExecutor-750-thread-1) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f04b1bb: rootName = solr_36013, domain = solr.core.legacyTrue.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.legacyTrue.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4cf84a26
[junit4] 2> 1081109 INFO (coreCloseExecutor-750-thread-1) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.legacyTrue.shard1.leader, tag=474549820
[junit4] 2> 1081111 INFO (zkCallback-411-thread-2-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1081201 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.c.Overseer Overseer (id=73265434021920771-127.0.0.1:36013_solr-n_0000000000) closing
[junit4] 2> 1081201 INFO (OverseerStateUpdate-73265434021920771-127.0.0.1:36013_solr-n_0000000000) [n:127.0.0.1:36013_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36013_solr
[junit4] 2> 1081317 WARN (OverseerAutoScalingTriggerThread-73265434021920771-127.0.0.1:36013_solr-n_0000000000) [n:127.0.0.1:36013_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1081370 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@13e56c7f{/solr,null,UNAVAILABLE}
[junit4] 2> 1081372 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1081421 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1081425 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1081425 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1081425 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1081427 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41fbd312{/solr,null,AVAILABLE}
[junit4] 2> 1081427 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@68c85bb4{HTTP/1.1,[http/1.1]}{127.0.0.1:36013}
[junit4] 2> 1081427 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.e.j.s.Server Started @1104277ms
[junit4] 2> 1081427 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36013}
[junit4] 2> 1081427 ERROR (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1081466 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1081466 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1081466 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1081466 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-09T12:11:13.985Z
[junit4] 2> 1081535 INFO (zkConnectionManagerCallback-422-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1081535 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1081613 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46602/solr
[junit4] 2> 1081714 INFO (zkConnectionManagerCallback-426-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1081852 INFO (zkConnectionManagerCallback-428-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1082065 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1082066 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36013_solr
[junit4] 2> 1082066 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.Overseer Overseer (id=73265434021920776-127.0.0.1:36013_solr-n_0000000001) starting
[junit4] 2> 1082155 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36013_solr
[junit4] 2> 1082156 INFO (zkCallback-419-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1082201 INFO (OverseerStateUpdate-73265434021920776-127.0.0.1:36013_solr-n_0000000001) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1082543 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1082624 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1082624 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1082662 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/.
[junit4] 2> 1082662 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [n:127.0.0.1:36013_solr ] o.a.s.c.CorePropertiesLocator Cores are: [legacyTrue_shard1_replica_n1]
[junit4] 2> 1082714 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1082860 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1082983 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.s.IndexSchema [legacyTrue_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1083038 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1083083 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1083137 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema [legacyTrue_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1083159 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1083159 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'legacyTrue_shard1_replica_n1' using configuration from collection legacyTrue, trusted=true
[junit4] 2> 1083160 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36013.solr.core.legacyTrue.shard1.replica_n1' (registry 'solr.core.legacyTrue.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d2ac7b9
[junit4] 2> 1083160 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1083160 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [[legacyTrue_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/legacyTrue_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_BEE0BAB69C20E091-001/tempDir-001/node1/./legacyTrue_shard1_replica_n1/data/]
[junit4] 2> 1083822 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1083822 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1083823 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1083823 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1083914 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c30e5f7[legacyTrue_shard1_replica_n1] main]
[junit4] 2> 1083922 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1083922 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1083923 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1083923 INFO (coreLoadExecutor-761-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591925304398446592
[junit4] 2> 1083972 INFO (searcherExecutor-762-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [legacyTrue_shard1_replica_n1] Registered new searcher Searcher@7c30e5f7[legacyTrue_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/ has no replicas
[junit4] 2> 1083983 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1083992 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36013/solr/legacyTrue_shard1_replica_n1/ shard1
[junit4] 2> 1084010 INFO (coreZkRegister-752-thread-1-processing-n:127.0.0.1:36013_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:36013_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1084112 INFO (zkCallback-427-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1084960 INFO (qtp1210985333-2327) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1085169 INFO (qtp1210985333-2327) [n:127.0.0.1:36013_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=209
[junit4] 2> 1085171 INFO (qtp1210985333-2329) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyFalse&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1085429 INFO (OverseerThreadFactory-759-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection legacyFalse
[junit4] 2> 1087234 INFO (qtp1210985333-2332) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyFalse_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyFalse&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1087243 ERROR (qtp1210985333-2332) [n:127.0.0.1:36013_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'legacyFalse_shard1_replica_n1': non legacy mode coreNodeName missing {collection.configName=conf, numShards=1, shard=shard1, collection=legacyFalse, replicaType=NRT}
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:989)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:455)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: non legacy mode coreNodeName missing {collection.configName=conf, numShards=1, shard=shard1, collection=legacyFalse, replicaType=NRT}
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:936)
[junit4] 2> ... 39 more
[junit4] 2>
[junit4] 2> 1087243 INFO (qtp1210985333-2332) [n:127.0.0.1:36013_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyFalse_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyFalse&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=400 QTime=9
[junit4] 2> 1087244 ERROR (OverseerThreadFactory-759-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:36013/solr
[junit4] 2> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36013/solr: Error CREATEing SolrCore 'legacyFalse_shard1_replica_n1': non legacy mode coreNodeName missing {collection.configName=conf, numShards=1, shard=shard1, collection=legacyFalse, replicaType=NRT}
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1087245 ERROR (OverseerThreadFactory-759-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Cleaning up collection [legacyFalse].
[junit4] 2> 1087246 INFO (OverseerThreadFactory-759-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1088062 INFO (OverseerThreadFactory-759-thread-1-processing-n:127.0.0.1:36013_solr) [n:127.0.0.1:36013_solr ] o.a.s.c.a.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [legacyFalse]
[junit4] 2> 1088066 INFO (qtp1210985333-2329) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1089442 INFO (OverseerCollectionConfigSetProcessor-73265434021920776-127.0.0.1:36013_solr-n_0000000001) [n:127.0.0.1:36013_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1120226 ERROR (qtp1210985333-2329) [n:127.0.0.1:36013_solr ] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE with timeout=30
[junit4] 2> 1120226 INFO (qtp1210985333-2329) [n:127.0.0.1:36013_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyFalse&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=35055
[junit4] 2> 1120243 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[BEE0BAB69C20E091]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSwitchLegacyCloud
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LegacyCloudClusterPropTest -Dtests.method=testCreate
[...truncated too long message...]
e/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/analysis/icu/lucene-analyzers-icu-7.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/contrib/analysis-extras/lib/icu4j-60.2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5718927179366285915000.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5713723369760884563014.suites -stdin
[junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5718698585797364681421.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5719149771284901948172.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:780: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1561: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1086: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5718698585797364681421.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180209_115248_5719149771284901948172.syserr
Total time: 432 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-7.x #359 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 359 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/359/
1 tests failed.
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey
Error Message:
There are still nodes recoverying - waited for 330 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
at __randomizedtesting.SeedInfo.seed([B346B10DE76EE80:8013B8C19F704504]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12038 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4] 2> 87513 INFO (SUITE-ShardSplitTest-seed#[B346B10DE76EE80]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/init-core-data-001
[junit4] 2> 87514 WARN (SUITE-ShardSplitTest-seed#[B346B10DE76EE80]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
[junit4] 2> 87522 INFO (SUITE-ShardSplitTest-seed#[B346B10DE76EE80]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 87524 INFO (SUITE-ShardSplitTest-seed#[B346B10DE76EE80]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 87524 INFO (SUITE-ShardSplitTest-seed#[B346B10DE76EE80]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 87575 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 87576 INFO (Thread-49) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 87576 INFO (Thread-49) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 87600 ERROR (Thread-49) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 87676 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkTestServer start zk server on port:40423
[junit4] 2> 87691 INFO (zkConnectionManagerCallback-75-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87732 INFO (zkConnectionManagerCallback-77-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 87746 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 87748 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 87750 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 87759 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 87760 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 87764 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 87779 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 87780 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 87782 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 87783 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 87784 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 87786 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 88070 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 88071 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 88071 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 88071 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 88072 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@73db76c5{/,null,AVAILABLE}
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4b04ae7a{HTTP/1.1,[http/1.1]}{127.0.0.1:35299}
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server Started @99447ms
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/tempDir-001/control/data, hostContext=/, hostPort=35299, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/control-001/cores}
[junit4] 2> 88074 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 88074 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 88075 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-08T08:36:10.925Z
[junit4] 2> 88077 INFO (zkConnectionManagerCallback-79-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88078 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 88078 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/control-001/solr.xml
[junit4] 2> 88083 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 88083 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 88104 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 88124 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40423/solr
[junit4] 2> 88130 INFO (zkConnectionManagerCallback-83-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88171 INFO (zkConnectionManagerCallback-85-thread-1-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88602 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 88602 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35299_
[junit4] 2> 88603 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.c.Overseer Overseer (id=72084881352425476-127.0.0.1:35299_-n_0000000000) starting
[junit4] 2> 88669 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35299_
[junit4] 2> 88698 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 88938 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 88994 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 89011 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 89013 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35299_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/control-001/cores
[junit4] 2> 89234 INFO (zkConnectionManagerCallback-91-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 89295 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 89303 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40423/solr ready
[junit4] 2> 89333 INFO (qtp317349255-303) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35299_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 89419 INFO (OverseerThreadFactory-124-thread-1-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 89558 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 89559 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 89731 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 90718 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 90899 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 91367 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 91923 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 91924 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 91997 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 91997 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 92048 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=45, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 92136 WARN (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 92502 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 92502 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 92532 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 92532 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 92535 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13683484628557097]
[junit4] 2> 92536 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6663ae4d[control_collection_shard1_replica_n1] main]
[junit4] 2> 92553 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 92553 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 92554 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 92623 INFO (searcherExecutor-127-thread-1-processing-n:127.0.0.1:35299_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@6663ae4d[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 92623 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591821179786625024
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35299/control_collection_shard1_replica_n1/
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:35299/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 92745 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 92812 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35299/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 92856 INFO (zkCallback-84-thread-2-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 92915 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 92917 INFO (qtp317349255-307) [n:127.0.0.1:35299_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3359
[junit4] 2> 92935 INFO (qtp317349255-303) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 93059 INFO (zkCallback-84-thread-3-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 93524 INFO (OverseerCollectionConfigSetProcessor-72084881352425476-127.0.0.1:35299_-n_0000000000) [n:127.0.0.1:35299_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 93941 INFO (qtp317349255-303) [n:127.0.0.1:35299_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35299_&wt=javabin&version=2} status=0 QTime=4607
[junit4] 2> 94059 INFO (zkConnectionManagerCallback-95-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 94099 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 94135 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40423/solr ready
[junit4] 2> 94135 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 94152 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 94242 INFO (OverseerThreadFactory-124-thread-2-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 94255 WARN (OverseerThreadFactory-124-thread-2-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 94507 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 94517 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=364
[junit4] 2> 95337 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001 of type NRT
[junit4] 2> 95338 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 95486 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 95487 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 95487 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 95496 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e311726{/,null,AVAILABLE}
[junit4] 2> 95496 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65268cd9{HTTP/1.1,[http/1.1]}{127.0.0.1:35548}
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server Started @106870ms
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=35548, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001/cores}
[junit4] 2> 95497 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 95497 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-08T08:36:18.348Z
[junit4] 2> 95543 INFO (zkConnectionManagerCallback-97-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 95569 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 95569 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001/solr.xml
[junit4] 2> 95607 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 95607 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 95608 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 95611 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40423/solr
[junit4] 2> 95662 INFO (zkConnectionManagerCallback-101-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 95675 INFO (zkConnectionManagerCallback-103-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 95690 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 95692 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 95728 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35548_
[junit4] 2> 95796 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 95815 INFO (zkCallback-84-thread-3-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 95868 INFO (zkCallback-94-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 96223 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 96298 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 96298 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 96312 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:35548_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001/cores
[junit4] 2> 96344 INFO (OverseerCollectionConfigSetProcessor-72084881352425476-127.0.0.1:35299_-n_0000000000) [n:127.0.0.1:35299_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 96512 INFO (qtp317349255-305) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:35548_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 96534 INFO (OverseerThreadFactory-124-thread-3-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:35548_ for creating new replica
[junit4] 2> 96666 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 96667 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 96730 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 97956 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 98071 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 98824 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 98970 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
[junit4] 2> 98971 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 98971 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 98971 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
[junit4] 2> 99023 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=45, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 99139 WARN (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 99578 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 99578 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 99612 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 99612 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 99614 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13683484628557097]
[junit4] 2> 99628 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d93e5ee[collection1_shard2_replica_n41] main]
[junit4] 2> 99630 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 99630 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 99631 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 99743 INFO (searcherExecutor-138-thread-1-processing-n:127.0.0.1:35548_ x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@6d93e5ee[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 99743 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591821187252486144
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35548/collection1_shard2_replica_n41/
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:35548/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 99865 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 99901 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35548/collection1_shard2_replica_n41/ shard2
[junit4] 2> 99919 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 99953 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 99955 INFO (qtp1934973156-352) [n:127.0.0.1:35548_ c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3288
[junit4] 2> 100028 INFO (qtp317349255-305) [n:127.0.0.1:35299_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:35548_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=3516
[junit4] 2> 100051 INFO (OverseerCollectionConfigSetProcessor-72084881352425476-127.0.0.1:35299_-n_0000000000) [n:127.0.0.1:35299_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 100102 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 102359 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001 of type NRT
[junit4] 2> 102359 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 102396 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 102396 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 102396 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 102397 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66d7c6a0{/,null,AVAILABLE}
[junit4] 2> 102398 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f98f10f{HTTP/1.1,[http/1.1]}{127.0.0.1:40084}
[junit4] 2> 102398 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server Started @113771ms
[junit4] 2> 102398 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=40084, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001/cores}
[junit4] 2> 102398 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 102421 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 102421 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 102422 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 102422 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-08T08:36:25.273Z
[junit4] 2> 102568 INFO (zkConnectionManagerCallback-107-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 102635 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 102636 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001/solr.xml
[junit4] 2> 102791 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 102791 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 102896 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 102932 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40423/solr
[junit4] 2> 103011 INFO (zkConnectionManagerCallback-111-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 103088 INFO (zkConnectionManagerCallback-113-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 103121 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 103140 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 103160 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40084_
[junit4] 2> 103161 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 103161 INFO (zkCallback-84-thread-3-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 103182 INFO (zkCallback-94-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 103248 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 103592 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 103637 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 103663 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 103679 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:40084_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001/cores
[junit4] 2> 103983 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:40084_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 104070 INFO (OverseerThreadFactory-124-thread-4-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:40084_ for creating new replica
[junit4] 2> 104182 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 104183 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 104385 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 104397 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 105474 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 105586 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
[junit4] 2> 106136 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 106391 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
[junit4] 2> 106391 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 106392 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 106392 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
[junit4] 2> 106428 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=45, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 106521 WARN (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 106722 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 106722 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 106735 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 106735 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 106737 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13683484628557097]
[junit4] 2> 106771 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@22979416[collection1_shard1_replica_n43] main]
[junit4] 2> 106780 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 106780 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 106781 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 106838 INFO (searcherExecutor-149-thread-1-processing-n:127.0.0.1:40084_ x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@22979416[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 106838 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591821194692132864
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40084/collection1_shard1_replica_n43/
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:40084/collection1_shard1_replica_n43/ has no replicas
[junit4] 2> 106856 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 106946 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40084/collection1_shard1_replica_n43/ shard1
[junit4] 2> 106954 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 106958 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 107011 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 107014 INFO (qtp1281940874-384) [n:127.0.0.1:40084_ c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2832
[junit4] 2> 107070 INFO (qtp317349255-307) [n:127.0.0.1:35299_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:40084_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=3087
[junit4] 2> 107129 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 107137 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 108102 INFO (OverseerCollectionConfigSetProcessor-72084881352425476-127.0.0.1:35299_-n_0000000000) [n:127.0.0.1:35299_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 108574 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001 of type NRT
[junit4] 2> 108575 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 108581 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 108581 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 108581 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 108581 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@14d203e{/,null,AVAILABLE}
[junit4] 2> 108582 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5734469f{HTTP/1.1,[http/1.1]}{127.0.0.1:38260}
[junit4] 2> 108582 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.e.j.s.Server Started @119955ms
[junit4] 2> 108582 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=38260, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001/cores}
[junit4] 2> 108582 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 108625 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 108625 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 108625 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 108626 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-08T08:36:31.476Z
[junit4] 2> 108656 INFO (zkConnectionManagerCallback-117-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 108676 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 108676 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001/solr.xml
[junit4] 2> 108732 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 108732 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 108734 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 108765 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40423/solr
[junit4] 2> 108857 INFO (zkConnectionManagerCallback-121-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 108958 INFO (zkConnectionManagerCallback-123-thread-1-processing-n:127.0.0.1:38260_) [n:127.0.0.1:38260_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 108985 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 109021 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 109041 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38260_
[junit4] 2> 109042 INFO (zkCallback-94-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 109042 INFO (zkCallback-84-thread-2-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 109044 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 109076 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 109077 INFO (zkCallback-122-thread-1-processing-n:127.0.0.1:38260_) [n:127.0.0.1:38260_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 109527 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 109599 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 109599 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 109601 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[B346B10DE76EE80]) [n:127.0.0.1:38260_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001/cores
[junit4] 2> 109878 INFO (qtp317349255-305) [n:127.0.0.1:35299_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:38260_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 109899 INFO (OverseerThreadFactory-124-thread-5-processing-n:127.0.0.1:35299_) [n:127.0.0.1:35299_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:38260_ for creating new replica
[junit4] 2> 109962 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 109963 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 110056 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 110069 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 110237 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 110655 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
[junit4] 2> 111425 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 111734 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
[junit4] 2> 111734 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'solr.core.collection1.shard2.replica_n45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@424d986d
[junit4] 2> 111734 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 111735 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [[collection1_shard2_replica_n45] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001/cores/collection1_shard2_replica_n45], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
[junit4] 2> 111771 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=45, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 111881 WARN (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 112241 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 112241 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 112242 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 112242 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 112446 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13683484628557097]
[junit4] 2> 112469 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.s.SolrIndexSearcher Opening [Searcher@276a05ea[collection1_shard2_replica_n45] main]
[junit4] 2> 112470 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 112470 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 112471 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 112490 INFO (searcherExecutor-160-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher Searcher@276a05ea[collection1_shard2_replica_n45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 112505 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591821200634413056
[junit4] 2> 112653 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n45
[junit4] 2> 112660 INFO (qtp1092462177-420) [n:127.0.0.1:38260_ c:collection1 s:shard2 x:collection1_shard2_replica_n45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2698
[junit4] 2> 112806 INFO (updateExecutor-118-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 112807 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 112818 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 112940 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ c:collection1 s:shard2 r:core_node42 x:collection1_shard2_replica_n41] o.a.s.c.S.Request [collection1_shard2_replica_n41] webapp= path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=54
[junit4] 2> 112956 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ c:collection1 s:shard2 r:core_node42 x:collection1_shard2_replica_n41] o.a.s.c.S.Request [collection1_shard2_replica_n41] webapp= path=/admin/ping params={wt=javabin&version=2} status=0 QTime=70
[junit4] 2> 112957 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n45]
[junit4] 2> 112957 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 112957 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n45] as recovering, leader is [http://127.0.0.1:35548/collection1_shard2_replica_n41/] and I am [http://127.0.0.1:38260/collection1_shard2_replica_n45/]
[junit4] 2> 113002 INFO (qtp317349255-305) [n:127.0.0.1:35299_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:38260_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=3123
[junit4] 2> 113189 INFO (recoveryExecutor-119-thread-1-processing-n:127.0.0.1:38260_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:38260_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:35548]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:38260_&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 113200 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:35548_) [n:127.0.0.1:35548_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 113200 INFO (zkCallback-122-thread-1-processing-n:127.0.0.1:38260_) [n:127.0.0.1:38260_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 113210 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 113227 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:38260_, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:38260","node_name":"127.0.0.1:38260_","state":"recovering","type":"NRT"}
[junit4] 2> 113227 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 113227 INFO (qtp1934973156-357) [n:127.0.0.1:35548_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:38260_&onlyIfLeaderActive=true&core=collection1_shard2_replica_n41&coreNodeName=core_node46&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=16
[junit4] 2> 113257 INFO (zkCallback-112-thread-1-processing-n:127.0.0.1:40084_) [n:127.0.0.1:40084_ ] o.a.s.c.c
[...truncated too long message...]
[junit4] 2> 1036197 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 39673
[junit4] 2> 1036197 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2132617887
[junit4] 2> 1036198 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1036198 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3fbfd833: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7a655964
[junit4] 2> 1036213 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1036213 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@626ca624: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@59f2ff36
[junit4] 2> 1036213 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1036213 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ad2bd91: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5dcd70d0
[junit4] 2> 1036287 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39673_
[junit4] 2> 1036333 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1036460 INFO (zkCallback-430-thread-5-processing-n:127.0.0.1:39673_) [n:127.0.0.1:39673_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1036545 INFO (coreCloseExecutor-853-thread-1) [n:127.0.0.1:39673_ c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.c.SolrCore [collection1_shard1_replica_t47] CLOSING SolrCore org.apache.solr.core.SolrCore@1b07beae
[junit4] 2> 1036545 INFO (coreCloseExecutor-853-thread-1) [n:127.0.0.1:39673_ c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_t47, tag=453492398
[junit4] 2> 1036545 INFO (coreCloseExecutor-853-thread-1) [n:127.0.0.1:39673_ c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3e108ef1: rootName = null, domain = solr.core.collection1.shard1.replica_t47, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_t47 / com.codahale.metrics.MetricRegistry@20649c96
[junit4] 2> 1036583 INFO (coreCloseExecutor-853-thread-1) [n:127.0.0.1:39673_ c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=453492398
[junit4] 2> 1036637 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.c.Overseer Overseer (id=72084908919816210-127.0.0.1:39673_-n_0000000004) closing
[junit4] 2> 1036637 WARN (OverseerAutoScalingTriggerThread-72084908919816210-127.0.0.1:39673_-n_0000000004) [n:127.0.0.1:39673_ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1036658 INFO (OverseerStateUpdate-72084908919816210-127.0.0.1:39673_-n_0000000004) [n:127.0.0.1:39673_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39673_
[junit4] 2> 1036715 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@4eb23d7f{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1036715 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6d5a8008{/,null,UNAVAILABLE}
[junit4] 2> 1036715 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1036716 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1036718 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[B346B10DE76EE80]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34225 34225
[junit4] 2> 1037017 INFO (Thread-287) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34225 34225
[junit4] 2> 1037045 WARN (Thread-287) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 6 /solr/collections/collection1/state.json
[junit4] 2> 6 /solr/configs/conf1
[junit4] 2> 6 /solr/collections/collection1/terms/shard1
[junit4] 2> 5 /solr/collections/collection1/terms/shard2
[junit4] 2> 3 /solr/collections/collection1/terms/shard1_0
[junit4] 2> 3 /solr/collections/collection1/terms/shard1_1
[junit4] 2> 2 /solr/collections/control_collection/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 2 /solr/overseer_elect/election/72084908919816201-127.0.0.1:43915_-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2> 3 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.method=testSplitWithChaosMonkey -Dtests.seed=B346B10DE76EE80 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-DO -Dtests.timezone=America/Argentina/Catamarca -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 529s J1 | ShardSplitTest.testSplitWithChaosMonkey <<<
[junit4] > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
[junit4] > at __randomizedtesting.SeedInfo.seed([B346B10DE76EE80:8013B8C19F704504]:0)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
[junit4] > at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:431)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_B346B10DE76EE80-001
[junit4] 2> Feb 08, 2018 8:51:59 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=FST50, shard_s=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=Asserting), id=FST50}, docValues:{rnd_b=DocValuesFormat(name=Direct), shard_s=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Asserting), n_ti=DocValuesFormat(name=Asserting), a_t=DocValuesFormat(name=Asserting), a_i=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=321, maxMBSortInHeap=5.596095507272875, sim=RandomSimilarity(queryNorm=false): {}, locale=es-DO, timezone=America/Argentina/Catamarca
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=110387728,total=463994880
[junit4] 2> NOTE: All tests run in this JVM: [XsltUpdateRequestHandlerTest, BlockJoinFacetSimpleTest, TestXIncludeConfig, TestPerFieldSimilarityWithDefaultOverride, ChaosMonkeySafeLeaderTest, ShardSplitTest]
[junit4] Completed [97/774 (1!)] on J1 in 950.10s, 5 tests, 1 failure <<< FAILURES!
[...truncated 50475 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 358 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/358/
4 tests failed.
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at http://127.0.0.1:42865/solr/awhollynewcollection_0: No registered leader was found after waiting for 4000ms , collection: awhollynewcollection_0 slice: shard2 saw state=DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/11)={ "pullReplicas":"0", "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{ "core_node3":{ "core":"awhollynewcollection_0_shard1_replica_n1", "base_url":"http://127.0.0.1:35360/solr", "node_name":"127.0.0.1:35360_solr", "state":"active", "type":"NRT", "leader":"true"}, "core_node5":{ "core":"awhollynewcollection_0_shard1_replica_n2", "base_url":"http://127.0.0.1:32956/solr", "node_name":"127.0.0.1:32956_solr", "state":"active", "type":"NRT"}, "core_node7":{ "core":"awhollynewcollection_0_shard1_replica_n4", "base_url":"http://127.0.0.1:42865/solr", "node_name":"127.0.0.1:42865_solr", "state":"active", "type":"NRT"}}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{ "core_node9":{ "core":"awhollynewcollection_0_shard2_replica_n6", "base_url":"http://127.0.0.1:37411/solr", "node_name":"127.0.0.1:37411_solr", "state":"down", "type":"NRT"}, "core_node11":{ "core":"awhollynewcollection_0_shard2_replica_n8", "base_url":"http://127.0.0.1:35360/solr", "node_name":"127.0.0.1:35360_solr", "state":"active", "type":"NRT"}, "core_node13":{ "core":"awhollynewcollection_0_shard2_replica_n10", "base_url":"http://127.0.0.1:32956/solr", "node_name":"127.0.0.1:32956_solr", "state":"active", "type":"NRT"}}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{ "core_node15":{ "core":"awhollynewcollection_0_shard3_replica_n12", "base_url":"http://127.0.0.1:42865/solr", "node_name":"127.0.0.1:42865_solr", "state":"active", "type":"NRT", "leader":"true"}, "core_node17":{ "core":"awhollynewcollection_0_shard3_replica_n14", "base_url":"http://127.0.0.1:37411/solr", "node_name":"127.0.0.1:37411_solr", "state":"active", "type":"NRT"}, "core_node18":{ "core":"awhollynewcollection_0_shard3_replica_n16", "base_url":"http://127.0.0.1:35360/solr", "node_name":"127.0.0.1:35360_solr", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"3", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"} with live_nodes=[127.0.0.1:32956_solr, 127.0.0.1:35360_solr, 127.0.0.1:42865_solr, 127.0.0.1:37411_solr]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42865/solr/awhollynewcollection_0: No registered leader was found after waiting for 4000ms , collection: awhollynewcollection_0 slice: shard2 saw state=DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/11)={
"pullReplicas":"0",
"replicationFactor":"3",
"shards":{
"shard1":{
"range":"80000000-d554ffff",
"state":"active",
"replicas":{
"core_node3":{
"core":"awhollynewcollection_0_shard1_replica_n1",
"base_url":"http://127.0.0.1:35360/solr",
"node_name":"127.0.0.1:35360_solr",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node5":{
"core":"awhollynewcollection_0_shard1_replica_n2",
"base_url":"http://127.0.0.1:32956/solr",
"node_name":"127.0.0.1:32956_solr",
"state":"active",
"type":"NRT"},
"core_node7":{
"core":"awhollynewcollection_0_shard1_replica_n4",
"base_url":"http://127.0.0.1:42865/solr",
"node_name":"127.0.0.1:42865_solr",
"state":"active",
"type":"NRT"}}},
"shard2":{
"range":"d5550000-2aa9ffff",
"state":"active",
"replicas":{
"core_node9":{
"core":"awhollynewcollection_0_shard2_replica_n6",
"base_url":"http://127.0.0.1:37411/solr",
"node_name":"127.0.0.1:37411_solr",
"state":"down",
"type":"NRT"},
"core_node11":{
"core":"awhollynewcollection_0_shard2_replica_n8",
"base_url":"http://127.0.0.1:35360/solr",
"node_name":"127.0.0.1:35360_solr",
"state":"active",
"type":"NRT"},
"core_node13":{
"core":"awhollynewcollection_0_shard2_replica_n10",
"base_url":"http://127.0.0.1:32956/solr",
"node_name":"127.0.0.1:32956_solr",
"state":"active",
"type":"NRT"}}},
"shard3":{
"range":"2aaa0000-7fffffff",
"state":"active",
"replicas":{
"core_node15":{
"core":"awhollynewcollection_0_shard3_replica_n12",
"base_url":"http://127.0.0.1:42865/solr",
"node_name":"127.0.0.1:42865_solr",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node17":{
"core":"awhollynewcollection_0_shard3_replica_n14",
"base_url":"http://127.0.0.1:37411/solr",
"node_name":"127.0.0.1:37411_solr",
"state":"active",
"type":"NRT"},
"core_node18":{
"core":"awhollynewcollection_0_shard3_replica_n16",
"base_url":"http://127.0.0.1:35360/solr",
"node_name":"127.0.0.1:35360_solr",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"3",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"} with live_nodes=[127.0.0.1:32956_solr, 127.0.0.1:35360_solr, 127.0.0.1:42865_solr, 127.0.0.1:37411_solr]
at __randomizedtesting.SeedInfo.seed([CB3598CF68DD4D97:8340EC7B6EEE6202]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:462)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([CB3598CF68DD4D97:7139AF4037359BD8]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([CB3598CF68DD4D97:33EB0D2710B19CC4]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:902)
at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=2=count(//lst[@name='segments']/lst/str[@name='version'][.='7.3.0'])
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><lst name="segments"><lst name="_0"><str name="name">_0</str><int name="delCount">1</int><long name="sizeInBytes">1766</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.391Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_1"><str name="name">_1</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.596Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_2"><str name="name">_2</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.679Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_3"><str name="name">_3</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.837Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_4"><str name="name">_4</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.839Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_5"><str name="name">_5</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.842Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst></lst>
</response>
request was:qt=/admin/segments&wt=xml
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:895)
... 40 more
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
Stack Trace:
java.lang.ClassCastException: org.apache.solr.cloud.autoscaling.SearchRateTrigger$SearchRateEvent cannot be cast to org.apache.solr.cloud.autoscaling.NodeAddedTrigger$NodeAddedEvent
at __randomizedtesting.SeedInfo.seed([CB3598CF68DD4D97:280DA6161BA8B62]:0)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11790 lines...]
[junit4] Suite: org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.handler.admin.SegmentsInfoRequestHandlerTest_CB3598CF68DD4D97-001/init-core-data-001
[junit4] 2> 226490 WARN (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 226509 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 226511 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 226512 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 226512 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 226513 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 226617 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 226753 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 227057 WARN (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 227069 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.4 with uniqueid field id
[junit4] 2> 227248 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2880ab8b
[junit4] 2> 227351 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2880ab8b
[junit4] 2> 227352 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2880ab8b
[junit4] 2> 227395 INFO (coreLoadExecutor-209-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 227395 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 227545 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 227694 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 228288 WARN (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 228289 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.4 with uniqueid field id
[junit4] 2> 228345 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 228346 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2880ab8b
[junit4] 2> 228346 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 228358 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.handler.admin.SegmentsInfoRequestHandlerTest_CB3598CF68DD4D97-001/init-core-data-001/]
[junit4] 2> 228725 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 228725 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 228727 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a14ebd6[collection1] main]
[junit4] 2> 228737 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 228838 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 229092 INFO (coreLoadExecutor-209-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 229092 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 229094 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 229144 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 229147 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 229147 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 229145 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR1000]} 0 50
[junit4] 2> 229150 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 229151 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 229233 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 229375 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 229375 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR1001]} 0 225
[junit4] 2> 229376 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 229377 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR1002]} 0 0
[junit4] 2> 229378 INFO (searcherExecutor-210-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5a14ebd6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 229392 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR1003]} 0 14
[junit4] 2> 229393 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR1004]} 0 0
[junit4] 2> 229461 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{delete=[SOLR1000]} 0 67
[junit4] 2> 229461 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 229462 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7623697f commitCommandVersion:0
[junit4] 2> 229600 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@79feb8e9[collection1] main]
[junit4] 2> 229601 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 229603 INFO (searcherExecutor-210-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79feb8e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C2/1:delGen=1) Uninverting(_1(7.3.0):C2) Uninverting(_2(7.3.0):C1)))}
[junit4] 2> 229603 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 141
[junit4] 2> 229619 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR2000]} 0 0
[junit4] 2> 229620 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR2001]} 0 1
[junit4] 2> 229621 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR2002]} 0 0
[junit4] 2> 229623 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR2003]} 0 1
[junit4] 2> 229624 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[SOLR2004]} 0 0
[junit4] 2> 229624 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 229624 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7623697f commitCommandVersion:0
[junit4] 2> 229626 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@72fe277c[collection1] main]
[junit4] 2> 229648 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 229649 INFO (searcherExecutor-210-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@72fe277c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C2/1:delGen=1) Uninverting(_1(7.3.0):C2) Uninverting(_2(7.3.0):C1) Uninverting(_3(7.3.0):C2) Uninverting(_4(7.3.0):C2) Uninverting(_5(7.3.0):C1)))}
[junit4] 2> 229649 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 25
[junit4] 2> 229694 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfos-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSegmentInfos
[junit4] 2> 229695 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfos-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={qt=/admin/segments&wt=xml} status=0 QTime=1
[junit4] 2> 229705 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfos-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSegmentInfos
[junit4] 2> 229718 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosData-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSegmentInfosData
[junit4] 2> 229719 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosData-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={qt=/admin/segments&wt=xml} status=0 QTime=0
[junit4] 2> 229720 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosData-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSegmentInfosData
[junit4] 2> 229754 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosVersion-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSegmentInfosVersion
[junit4] 2> 229755 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosVersion-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={qt=/admin/segments&wt=xml} status=0 QTime=0
[junit4] 2> 229756 ERROR (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosVersion-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=2=count(//lst[@name='segments']/lst/str[@name='version'][.='7.3.0'])
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><lst name="segments"><lst name="_0"><str name="name">_0</str><int name="delCount">1</int><long name="sizeInBytes">1766</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.391Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_1"><str name="name">_1</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.596Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_2"><str name="name">_2</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.679Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_3"><str name="name">_3</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.837Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_4"><str name="name">_4</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.839Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_5"><str name="name">_5</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.842Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst></lst>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:qt=/admin/segments&wt=xml
[junit4] 2> 229757 ERROR (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosVersion-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: qt=/admin/segments&wt=xml:java.lang.RuntimeException: REQUEST FAILED: xpath=2=count(//lst[@name='segments']/lst/str[@name='version'][.='7.3.0'])
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><lst name="segments"><lst name="_0"><str name="name">_0</str><int name="delCount">1</int><long name="sizeInBytes">1766</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.391Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_1"><str name="name">_1</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.596Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_2"><str name="name">_2</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.679Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_3"><str name="name">_3</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.837Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_4"><str name="name">_4</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.839Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_5"><str name="name">_5</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.842Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst></lst>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:qt=/admin/segments&wt=xml
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:895)
[junit4] 2> at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 229757 INFO (TEST-SegmentsInfoRequestHandlerTest.testSegmentInfosVersion-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSegmentInfosVersion
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SegmentsInfoRequestHandlerTest -Dtests.method=testSegmentInfosVersion -Dtests.seed=CB3598CF68DD4D97 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=he -Dtests.timezone=Asia/Chita -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.04s J0 | SegmentsInfoRequestHandlerTest.testSegmentInfosVersion <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4] > at __randomizedtesting.SeedInfo.seed([CB3598CF68DD4D97:33EB0D2710B19CC4]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:902)
[junit4] > at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=2=count(//lst[@name='segments']/lst/str[@name='version'][.='7.3.0'])
[junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] > <response>
[junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><lst name="segments"><lst name="_0"><str name="name">_0</str><int name="delCount">1</int><long name="sizeInBytes">1766</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.391Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_1"><str name="name">_1</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.596Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_2"><str name="name">_2</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.679Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_3"><str name="name">_3</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.837Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_4"><str name="name">_4</str><int name="delCount">0</int><long name="sizeInBytes">1699</long><int name="size">2</int><date name="age">2018-02-07T04:41:43.839Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst><lst name="_5"><str name="name">_5</str><int name="delCount">0</int><long name="sizeInBytes">1658</long><int name="size">1</int><date name="age">2018-02-07T04:41:43.842Z</date><str name="source">flush</str><str name="version">7.3.0</str></lst></lst>
[junit4] > </response>
[junit4] > request was:qt=/admin/segments&wt=xml
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:895)
[junit4] > ... 40 more
[junit4] 2> 229759 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 229759 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1455245927
[junit4] 2> 229759 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 229759 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@27ebccaf: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6d5b0a6b
[junit4] 2> 229778 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 229779 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@46590431: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@585c3643
[junit4] 2> 229781 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 229781 INFO (SUITE-SegmentsInfoRequestHandlerTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@18cbaf3: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5c63a990
[junit4] 2> 229827 INFO (coreCloseExecutor-214-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@48f5efec
[junit4] 2> 229828 INFO (coreCloseExecutor-214-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1224077292
[junit4] 2> 229828 INFO (coreCloseExecutor-214-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2a3bae48: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@c858d9d
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.handler.admin.SegmentsInfoRequestHandlerTest_CB3598CF68DD4D97-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=he, timezone=Asia/Chita
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=188027776,total=339214336
[junit4] 2> NOTE: All tests run in this JVM: [HdfsBasicDistributedZk2Test, TestGroupingSearch, ChaosMonkeySafeLeaderWithPullReplicasTest, ParsingFieldUpdateProcessorsTest, MetricsHandlerTest, SolrPluginUtilsTest, TestLegacyNumericUtils, TestRemoteStreaming, ShufflingReplicaListTransformerTest, TestCollectionAPIs, ZkSolrClientTest, ExternalFileFieldSortTest, TestSimpleQParserPlugin, SegmentsInfoRequestHandlerTest]
[junit4] Completed [30/774 (1!)] on J0 in 3.46s, 3 tests, 1 error <<< FAILURES!
[...truncated 153 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_CB3598CF68DD4D97-001/init-core-data-001
[junit4] 2> 442931 WARN (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 442932 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 442933 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 442933 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 442934 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_CB3598CF68DD4D97-001/tempDir-001
[junit4] 2> 442934 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 442934 INFO (Thread-845) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 442934 INFO (Thread-845) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 442936 ERROR (Thread-845) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 443034 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42984
[junit4] 2> 443040 INFO (zkConnectionManagerCallback-448-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443075 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 443075 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@792128ab{/solr,null,AVAILABLE}
[junit4] 2> 443076 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@723783f3{/solr,null,AVAILABLE}
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@660b953b{HTTP/1.1,[http/1.1]}{127.0.0.1:36012}
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.e.j.s.Server Started @454540ms
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36012}
[junit4] 2> 443078 ERROR (jetty-launcher-445-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 443078 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-07T04:45:16.262Z
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2939ce6b{HTTP/1.1,[http/1.1]}{127.0.0.1:40703}
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.e.j.s.Server Started @454541ms
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40703}
[junit4] 2> 443079 ERROR (jetty-launcher-445-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 443079 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-07T04:45:16.263Z
[junit4] 2> 443088 INFO (zkConnectionManagerCallback-450-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443089 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 443089 INFO (zkConnectionManagerCallback-452-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443098 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 443099 INFO (jetty-launcher-445-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42984/solr
[junit4] 2> 443104 INFO (zkConnectionManagerCallback-456-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443132 INFO (jetty-launcher-445-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42984/solr
[junit4] 2> 443144 INFO (zkConnectionManagerCallback-460-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443146 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10012d790400004, likely client has closed socket
[junit4] 2> 443156 INFO (zkConnectionManagerCallback-462-thread-1-processing-n:127.0.0.1:36012_solr) [n:127.0.0.1:36012_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443158 INFO (zkConnectionManagerCallback-464-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443288 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 443288 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 443288 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40703_solr
[junit4] 2> 443289 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36012_solr
[junit4] 2> 443299 INFO (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 443311 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.Overseer Overseer (id=72078311085309958-127.0.0.1:40703_solr-n_0000000000) starting
[junit4] 2> 443313 INFO (zkCallback-461-thread-1-processing-n:127.0.0.1:36012_solr) [n:127.0.0.1:36012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 443343 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 443344 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 443345 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40703_solr
[junit4] 2> 443345 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:36012_solr]
[junit4] 2> 443345 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 443345 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 443346 INFO (zkCallback-461-thread-1-processing-n:127.0.0.1:36012_solr) [n:127.0.0.1:36012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 443346 INFO (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 443352 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 443352 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 443352 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36012.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443353 DEBUG (ScheduledTrigger-866-thread-1) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 443371 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36012.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443374 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36012.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443375 INFO (jetty-launcher-445-thread-1) [n:127.0.0.1:36012_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_CB3598CF68DD4D97-001/tempDir-001/node1/.
[junit4] 2> 443438 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40703.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443448 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40703.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443448 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40703.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443449 INFO (jetty-launcher-445-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_CB3598CF68DD4D97-001/tempDir-001/node2/.
[junit4] 2> 443485 INFO (zkConnectionManagerCallback-470-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443489 INFO (zkConnectionManagerCallback-474-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443490 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 443491 INFO (SUITE-TriggerIntegrationTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42984/solr ready
[junit4] 2> 443519 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
[junit4] 2> 443519 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 443520 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 443520 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 443524 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 443525 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 443525 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 443563 INFO (qtp625173310-2414) [n:127.0.0.1:36012_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=3
[junit4] 2> 443566 INFO (qtp1215712408-2407) [n:127.0.0.1:40703_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 443568 DEBUG (qtp625173310-2406) [n:127.0.0.1:36012_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 443569 INFO (qtp625173310-2406) [n:127.0.0.1:36012_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=33
[junit4] 2> 443569 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 443572 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:36012_solr, 127.0.0.1:40703_solr]
[junit4] 2> 443572 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 443572 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 443573 INFO (qtp625173310-2416) [n:127.0.0.1:36012_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 443574 INFO (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 443584 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 443575 DEBUG (ScheduledTrigger-866-thread-1) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 443585 DEBUG (ScheduledTrigger-866-thread-1) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 443585 INFO (qtp1215712408-2411) [n:127.0.0.1:40703_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 443586 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 443586 DEBUG (qtp625173310-2406) [n:127.0.0.1:36012_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 443587 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 443588 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:36012_solr, 127.0.0.1:40703_solr]
[junit4] 2> 443588 DEBUG (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 443588 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 443588 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 443589 DEBUG (OverseerAutoScalingTriggerThread-72078311085309958-127.0.0.1:40703_solr-n_0000000000) [n:127.0.0.1:40703_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 443593 INFO (qtp625173310-2406) [n:127.0.0.1:36012_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 443596 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 443627 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 443627 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 443627 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 443627 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7d794ad8{/solr,null,AVAILABLE}
[junit4] 2> 443628 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61b5a69d{HTTP/1.1,[http/1.1]}{127.0.0.1:43760}
[junit4] 2> 443628 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.Server Started @455090ms
[junit4] 2> 443628 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43760}
[junit4] 2> 443628 ERROR (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 443634 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 443634 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 443634 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 443635 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-07T04:45:16.819Z
[junit4] 2> 443675 INFO (zkConnectionManagerCallback-476-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443676 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 443688 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42984/solr
[junit4] 2> 443690 INFO (zkConnectionManagerCallback-480-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443695 INFO (zkConnectionManagerCallback-482-thread-1-processing-n:127.0.0.1:43760_solr) [n:127.0.0.1:43760_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 443704 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 443711 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 443714 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43760_solr
[junit4] 2> 443719 INFO (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 443720 INFO (zkCallback-461-thread-1-processing-n:127.0.0.1:36012_solr) [n:127.0.0.1:36012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 443720 INFO (zkCallback-481-thread-1-processing-n:127.0.0.1:43760_solr) [n:127.0.0.1:43760_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 443722 INFO (zkCallback-473-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 443767 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43760.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443783 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43760.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443783 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43760.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75772c4
[junit4] 2> 443786 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:43760_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_CB3598CF68DD4D97-001/tempDir-001/node3/.
[junit4] 2> 444587 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 444587 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 444587 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:43760_solr at time 316118708594687
[junit4] 2> 445588 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 445588 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 445589 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:43760_solr] added at times [316118708594687], now=316119710091334
[junit4] 2> 445590 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"11f821f5527ffT10bl7ai0avmnlyml01fyz4xba",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":316118708594687,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[316118708594687],
[junit4] 2> "nodeNames":["127.0.0.1:43760_solr"]}}
[junit4] 2> 445591 DEBUG (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_cooldown_trigger]
[junit4] 2> 445599 INFO (ScheduledTrigger-866-thread-2) [n:127.0.0.1:40703_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 445602 DEBUG (AutoscalingActionExecutor-867-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"11f821f5527ffT10bl7ai0avmnlyml01fyz4xba",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":316118708594687,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[316118708594687],
[junit4] 2> "_enqueue_time_":316119712782474,
[junit4] 2> "nodeNames":["127.0.0.1:43760_solr"]}}
[junit4] 2> 445610 DEBUG (AutoscalingActionExecutor-867-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 5000ms
[junit4] 2> 445610 DEBUG (AutoscalingActionExecutor-867-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 8 ms for event id=11f821f5527ffT10bl7ai0avmnlyml01fyz4xba
[junit4] 2> 446605 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 446607 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 446607 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 446607 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 446608 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@373a7229{/solr,null,AVAILABLE}
[junit4] 2> 446608 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@32fa02cc{HTTP/1.1,[http/1.1]}{127.0.0.1:41290}
[junit4] 2> 446608 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.e.j.s.Server Started @458070ms
[junit4] 2> 446608 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41290}
[junit4] 2> 446608 ERROR (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 446608 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 446609 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 446609 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 446609 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-07T04:45:19.793Z
[junit4] 2> 446611 INFO (zkConnectionManagerCallback-486-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446612 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 446630 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42984/solr
[junit4] 2> 446636 INFO (zkConnectionManagerCallback-490-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446652 INFO (zkConnectionManagerCallback-492-thread-1-processing-n:127.0.0.1:41290_solr) [n:127.0.0.1:41290_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446662 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:41290_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 446665 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:41290_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 446669 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[CB3598CF68DD4D97]) [n:127.0.0.1:41290_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41290_solr
[junit4] 2> 446672 INFO (zkCallback-461-thread-1-processing-n:127.0.0.1:36012_solr) [n:127.0.0.1:36012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 446672 INFO (zkCallback-473-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 446672 INFO (zkCallback-481-thread-1-processing-n:127.0.0.1:43760_solr) [n:127.0.0.1:43760_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 446672 INFO (zkCallback-463-thread-1-processing-n:127.0.0.1:40703_solr) [n:127.0.0.1:40703_solr
[...truncated too long message...]
}
[junit4] 2> 4059582 INFO (jetty-closer-6389-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4059597 ERROR (SUITE-CollectionsAPIDistributedZkTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4059597 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[CB3598CF68DD4D97]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44837 44837
[junit4] 2> 4059763 INFO (Thread-4912) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44837 44837
[junit4] 2> 4059763 WARN (Thread-4912) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/aliases.json
[junit4] 2> 25 /solr/clusterprops.json
[junit4] 2> 22 /solr/configs/conf
[junit4] 2> 21 /solr/collections/addReplicaColl/terms/shard1
[junit4] 2> 15 /solr/collections/awhollynewcollection_0/terms/shard3
[junit4] 2> 14 /solr/collections/awhollynewcollection_0/terms/shard2
[junit4] 2> 13 /solr/collections/awhollynewcollection_0/terms/shard1
[junit4] 2> 9 /solr/collections/reloaded_collection/terms/shard1
[junit4] 2> 8 /solr/collections/nodes_used_collection/terms/shard1
[junit4] 2> 8 /solr/collections/reloaded_collection/terms/shard2
[junit4] 2> 7 /solr/collections/acollectionafterbaddelete/terms/shard1
[junit4] 2> 7 /solr/collections/nodes_used_collection/terms/shard2
[junit4] 2> 7 /solr/collections/addReplicaColl/terms/shard2
[junit4] 2> 6 /solr/configs/conf2
[junit4] 2> 5 /solr/security.json
[junit4] 2> 4 /solr/collections/halfdeletedcollection/terms/shard1
[junit4] 2> 3 /solr/collections/withconfigset2/terms/shard1
[junit4] 2> 3 /solr/collections/halfcollectionblocker/terms/shard1
[junit4] 2> 3 /solr/collections/onlyinzk/terms/shard2
[junit4] 2> 2 /solr/collections/created_and_deleted/terms/shard1
[junit4] 2> 2 /solr/collections/halfcollectionblocker2/terms/shard1
[junit4] 2> 2 /solr/collections/halfcollection/terms/shard2
[junit4] 2> 2 /solr/collections/halfdeletedcollection/terms/shard2
[junit4] 2> 2 /solr/collections/onlyinzk/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 59 /solr/collections/awhollynewcollection_0/state.json
[junit4] 2> 52 /solr/collections/addReplicaColl/state.json
[junit4] 2> 34 /solr/collections/reloaded_collection/state.json
[junit4] 2> 25 /solr/clusterstate.json
[junit4] 2> 24 /solr/collections/nodes_used_collection/state.json
[junit4] 2> 14 /solr/collections/halfdeletedcollection/state.json
[junit4] 2> 13 /solr/collections/acollectionafterbaddelete/state.json
[junit4] 2> 10 /solr/collections/onlyinzk/state.json
[junit4] 2> 5 /solr/collections/halfcollectionblocker/state.json
[junit4] 2> 5 /solr/collections/created_and_deleted/state.json
[junit4] 2> 5 /solr/collections/halfcollectionblocker2/state.json
[junit4] 2> 5 /solr/collections/withconfigset2/state.json
[junit4] 2> 4 /solr/collections/halfcollection/state.json
[junit4] 2> 2 /solr/collections/awhollynewcollection_0/leader_elect/shard1/election/72078541330907146-core_node3-n_0000000000
[junit4] 2> 2 /solr/collections/awhollynewcollection_0/leader_elect/shard2/election/72078541330907146-core_node11-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/72078541330907146-127.0.0.1:35360_solr-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/72078541330907147-127.0.0.1:42865_solr-n_0000000000
[junit4] 2> 2 /solr/collections/awhollynewcollection_0/leader_elect/shard3/election/72078541330907146-core_node18-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 312 /solr/collections
[junit4] 2> 31 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_CB3598CF68DD4D97-001
[junit4] 2> Feb 07, 2018 5:45:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=sk, timezone=Asia/Bangkok
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=85578768,total=518520832
[junit4] 2> NOTE: All tests run in this JVM: [TestRebalanceLeaders, TestSolr4Spatial, BadComponentTest, TestComplexPhraseLeadingWildcard, TestRuleBasedAuthorizationPlugin, ZkControllerTest, TestUseDocValuesAsStored, DeleteNodeTest, BaseCdcrDistributedZkTest, SolrShardReporterTest, SolrInfoBeanTest, TestHashQParserPlugin, TestImpersonationWithHadoopAuth, TestFoldingMultitermQuery, TestNRTOpen, CdcrReplicationDistributedZkTest, ResourceLoaderTest, DistributedQueryComponentOptimizationTest, TestQueryUtils, JvmMetricsTest, DistributedFacetSimpleRefinementLongTailTest, DocValuesNotIndexedTest, TestHdfsUpdateLog, TestFieldCache, BasicZkTest, PathHierarchyTokenizerFactoryTest, LegacyCloudClusterPropTest, AtomicUpdateProcessorFactoryTest, BJQParserTest, PropertiesRequestHandlerTest, TriggerIntegrationTest, SolrJmxReporterTest, RequiredFieldsTest, UpdateLogTest, DocExpirationUpdateProcessorFactoryTest, TestPostingsSolrHighlighter, AlternateDirectoryTest, XsltUpdateRequestHandlerTest, TestLRUCache, MetricTriggerTest, EnumFieldTest, PKIAuthenticationIntegrationTest, TestReversedWildcardFilterFactory, TestConfigSetsAPIExclusivity, MetricUtilsTest, LoggingHandlerTest, ShardSplitTest, TestExclusionRuleCollectionAccess, CdcrVersionReplicationTest, TestChildDocTransformer, TestPullReplicaErrorHandling, TestLegacyTerms, SoftAutoCommitTest, SampleTest, TestCoreBackup, TestReqParamsAPI, TestNamedUpdateProcessors, TriLevelCompositeIdRoutingTest, TestFieldSortValues, TestManagedSchemaAPI, SolrMetricManagerTest, TestOnReconnectListenerSupport, TestCorePropertiesReload, SolrCloudExampleTest, TestDistributedStatsComponentCardinality, HighlighterTest, ReplaceNodeNoTargetTest, TestReload, IndexSchemaTest, AutoscalingHistoryHandlerTest, BlockCacheTest, TestPivotHelperCode, BasicFunctionalityTest, TestUninvertingReader, MoveReplicaHDFSFailoverTest, PrimUtilsTest, InfixSuggestersTest, TestDynamicFieldCollectionResource, HdfsNNFailoverTest, DistribJoinFromCollectionTest, FieldMutatingUpdateProcessorTest, CachingDirectoryFactoryTest, DistributedDebugComponentTest, SchemaVersionSpecificBehaviorTest, CSVRequestHandlerTest, TestMaxScoreQueryParser, TestTrieFacet, TestCursorMarkWithoutUniqueKey, SignatureUpdateProcessorFactoryTest, TestDistributedGrouping, HdfsRestartWhileUpdatingTest, SaslZkACLProviderTest, SolrIndexMetricsTest, TestTrie, HttpSolrCallGetCoreTest, DirectoryFactoryTest, TestXIncludeConfig, TestLeaderInitiatedRecoveryThread, PeerSyncReplicationTest, AddReplicaTest, TestBulkSchemaConcurrent, ConcurrentDeleteAndCreateCollectionTest, TestMinMaxOnMultiValuedField, TestRestManager, HdfsDirectoryTest, DirectUpdateHandlerOptimizeTest, TestSurroundQueryParser, SmileWriterTest, OverriddenZkACLAndCredentialsProvidersTest, TestSolrCloudSnapshots, TestElisionMultitermQuery, TestImplicitCoreProperties, ClassificationUpdateProcessorIntegrationTest, PeerSyncTest, TestClassicSimilarityFactory, UpdateRequestProcessorFactoryTest, SolrCoreTest, ExecutePlanActionTest, TestMacroExpander, TestEmbeddedSolrServerAdminHandler, DOMUtilTest, TestFunctionQuery, SortByFunctionTest, RestartWhileUpdatingTest, TestCloudDeleteByQuery, CreateRoutedAliasTest, ClusterStateUpdateTest, TestRandomFaceting, TestDistributedSearch, TestJoin, TestCoreContainer, StatsComponentTest, SpellCheckComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SuggesterFSTTest, CoreAdminHandlerTest, NoCacheHeaderTest, TestUpdate, QueryEqualityTest, DistanceFunctionTest, DebugComponentTest, DocumentAnalysisRequestHandlerTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestIndexingPerformance, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, ReturnFieldsTest, TestAnalyzedSuggestions, JsonLoaderTest, TestPartialUpdateDeduplication, SearchHandlerTest, SOLR749Test, ResponseLogComponentTest, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, TestSearchPerf, NumericFieldsTest, TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, ScriptEngineTest, TestFastLRUCache, DateFieldTest, TestSolrJ, TestUtils, SliceStateTest, SystemInfoHandlerTest, CircularListTest, DistributedMLTComponentTest, DistributedIntervalFacetingTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionsAPISolrJTest, HttpPartitionTest, ReplicationFactorTest, SSLMigrationTest, TestAuthenticationFramework, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestLockTree, TestMiniSolrCloudClusterSSL, TestRandomRequestDistribution, TestSSLRandomization, TestSizeLimitedDistributedMap, TestStressLiveNodes, TestTlogReplica, CollectionTooManyReplicasTest, CollectionsAPIDistributedZkTest]
[junit4] Completed [579/774 (3!)] on J1 in 103.88s, 19 tests, 1 error <<< FAILURES!
[...truncated 48848 lines...]