You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/06/04 11:10:54 UTC
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 71 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/71/
4 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:53486/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:56607/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53486/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:56607/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([D82F6AFECC9977B:A74F255D5B1A42AB]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56607/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 47 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode
Error Message:
no STARTED event
Stack Trace:
java.lang.AssertionError: no STARTED event
at __randomizedtesting.SeedInfo.seed([D82F6AFECC9977B:AA6DEB0C23841863]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
there should be new MOVERPLICA ops
Stack Trace:
java.lang.AssertionError: there should be new MOVERPLICA ops
at __randomizedtesting.SeedInfo.seed([D82F6AFECC9977B:A678EBBA33151155]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
no STARTED event: [], waitFor=5, killDelay=5000, minIgnored=0
Stack Trace:
java.lang.AssertionError: no STARTED event: [], waitFor=5, killDelay=5000, minIgnored=0
at __randomizedtesting.SeedInfo.seed([D82F6AFECC9977B:B29738516F23F2FD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:456)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:323)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12858 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_D82F6AFECC9977B-001/init-core-data-001
[junit4] 2> 899858 INFO (SUITE-TestLargeCluster-seed#[D82F6AFECC9977B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 899859 INFO (SUITE-TestLargeCluster-seed#[D82F6AFECC9977B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 899860 INFO (SUITE-TestLargeCluster-seed#[D82F6AFECC9977B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 899877 DEBUG (SUITE-TestLargeCluster-seed#[D82F6AFECC9977B]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10005_solr
[junit4] 2> 899877 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 899877 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 899878 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 899878 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 899887 WARN (SUITE-TestLargeCluster-seed#[D82F6AFECC9977B]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 899907 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10040_solr, 127.0.0.1:10043_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10046_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr, 127.0.0.1:10007_solr, 127.0.0.1:10015_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10005_solr, 127.0.0.1:10013_solr, 127.0.0.1:10021_solr, 127.0.0.1:10035_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10008_solr, 127.0.0.1:10019_solr, 127.0.0.1:10024_solr, 127.0.0.1:10027_solr, 127.0.0.1:10022_solr, 127.0.0.1:10039_solr, 127.0.0.1:10033_solr, 127.0.0.1:10025_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10036_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10006_solr, 127.0.0.1:10045_solr, 127.0.0.1:10037_solr, 127.0.0.1:10041_solr, 127.0.0.1:10042_solr, 127.0.0.1:10009_solr, 127.0.0.1:10029_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10048_solr, 127.0.0.1:10011_solr]
[junit4] 2> 899921 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 899921 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 899921 DEBUG (ScheduledTrigger-2556-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 98
[junit4] 2> 899940 INFO (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
[junit4] 2> 899940 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 899940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 899941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 899941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 899941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 899941 INFO (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 899941 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 899942 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 899943 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 899943 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 899943 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 899943 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 899943 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[D82F6AFECC9977B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 899951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 899951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 899952 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 899952 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 899960 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 899975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 899975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 899976 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 899997 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900018 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900038 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900058 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900068 DEBUG (simCloudManagerPool-2560-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 900069 DEBUG (simCloudManagerPool-2560-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 900074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 900074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 900074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 900078 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900099 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900107 DEBUG (simCloudManagerPool-2560-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 900107 DEBUG (simCloudManagerPool-2560-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 900118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 900119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 900119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=compute, class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestLargeCluster$TestTriggerAction}], enabled=true}
[junit4] 2> 900120 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 900120 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900120 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900120 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900140 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900141 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900141 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900209 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900209 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900209 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900229 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900249 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900269 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900290 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900316 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900316 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900316 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900336 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900336 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900336 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900357 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900357 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900357 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900377 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900377 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900377 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900397 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900397 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900397 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900417 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900417 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900418 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900438 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900438 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900438 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900466 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900466 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900466 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900486 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900486 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900486 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900506 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900506 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900506 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900531 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900531 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900531 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900551 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900551 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900551 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900571 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900571 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900571 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900591 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900592 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900592 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900615 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900615 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900615 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900710 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900710 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900710 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900731 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900731 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900731 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900754 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900754 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900754 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900774 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900774 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900775 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900795 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900795 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900795 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900815 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900815 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900815 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900835 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900835 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900835 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900856 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900856 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900856 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900876 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900876 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900876 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900896 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900896 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900896 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 900916 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 900917 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 900917 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901093 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901093 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901093 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901113 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901113 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901113 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901133 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901133 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901133 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901154 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901154 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901154 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901174 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901174 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901174 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901194 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901194 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901194 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901214 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901214 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901214 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901235 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901235 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901235 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901255 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901255 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901255 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901275 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901275 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901275 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901295 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901295 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901295 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901316 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901316 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901316 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901336 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901336 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901336 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901356 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901356 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901356 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901376 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901376 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901377 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901397 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901397 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901397 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901417 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901417 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901417 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901437 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901437 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901437 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901457 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901458 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901458 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901478 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901478 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901478 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901498 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901498 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901498 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901518 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901518 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901518 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901539 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901539 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901539 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901559 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901559 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901559 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901579 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901579 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901579 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901600 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901600 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901600 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901620 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901620 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901620 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901640 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901640 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901640 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901660 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901660 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901660 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901681 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901681 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901681 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901701 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901701 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901701 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901721 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901721 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901721 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901743 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901743 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901743 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901763 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901763 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901763 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901787 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901787 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901787 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901807 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901808 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901808 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901828 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901828 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901828 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901848 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901848 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901848 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901870 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901871 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901871 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901891 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901891 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901891 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901911 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901911 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901912 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901932 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901932 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901932 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901952 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901952 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901952 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901972 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901972 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901973 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 901993 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 901993 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 901993 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902013 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902013 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902013 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902034 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902034 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902034 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902054 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902054 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902054 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902074 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902074 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902074 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902094 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902095 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902095 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902115 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902115 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902115 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902135 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902135 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902135 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902155 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902155 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902156 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902176 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902176 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902176 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902196 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902196 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902196 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902216 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902216 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902216 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902236 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902236 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902237 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902257 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902257 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902257 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902277 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902277 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902277 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902297 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902297 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902297 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902317 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902317 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902318 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902338 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902338 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902338 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902358 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902358 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902358 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902378 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902378 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902378 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902399 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902399 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902399 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902419 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902419 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902419 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902439 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902439 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902439 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902459 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902459 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902459 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902480 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902480 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902480 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902500 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902500 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902500 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902520 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902520 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902520 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902540 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902540 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902540 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902561 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902561 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902561 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902581 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902581 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902581 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902601 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902601 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902601 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902621 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902621 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902621 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902642 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902642 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902642 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902662 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902662 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902662 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902682 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902683 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902683 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902703 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902703 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902703 DEBUG (ScheduledTrigger-2561-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902724 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902724 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902724 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902745 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902745 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902745 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902765 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902765 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902765 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902785 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902785 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902785 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902805 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902805 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902805 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902826 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902826 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902826 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902846 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902846 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902846 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902866 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902866 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902866 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902886 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902886 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902887 DEBUG (ScheduledTrigger-2561-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902907 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902907 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902907 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902927 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902927 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902927 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902947 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902947 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902947 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902967 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902968 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902968 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 902988 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 902988 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 902988 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903008 DEBUG (ScheduledTrigger-2561-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 903008 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903008 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903028 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903028 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903029 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 903049 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903049 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903049 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 903069 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903069 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903069 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 903089 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903089 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903089 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 903109 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 903109 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 903110 DEBUG (ScheduledTrigger-2561-thread-1) [ ] o.a.s.c.a.No
[...truncated too long message...]
= /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1287290686
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1287290686
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1287290686
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1300137973
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1300137973
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1300137973
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 277ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 176 minutes 19 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 76 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/76/
6 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPISolrJTest.testAddAndDeleteReplica
Error Message:
Could not load collection from ZK: solrj_replicatests
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: solrj_replicatests
at __randomizedtesting.SeedInfo.seed([F64145A395D086EC:18F13CF6921C9156]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1316)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:732)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:117)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:256)
at org.apache.solr.cloud.CollectionsAPISolrJTest.testAddAndDeleteReplica(CollectionsAPISolrJTest.java:350)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/solrj_replicatests/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:341)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1328)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1314)
... 45 more
FAILED: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
Error Message:
should be at least one inactive event
Stack Trace:
java.lang.AssertionError: should be at least one inactive event
at __randomizedtesting.SeedInfo.seed([F64145A395D086EC:EB6D85D1F493A1E7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode
Error Message:
trigger did not fire
Stack Trace:
java.lang.AssertionError: trigger did not fire
at __randomizedtesting.SeedInfo.seed([F64145A395D086EC:51AE58005A9D09F4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:301)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([F64145A395D086EC:AB095B2A5A1620A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:636)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
last state: DocCollection(testNodeLost//clusterstate.json/463)={ "replicationFactor":"10", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"5", "autoAddReplicas":"false", "nrtReplicas":"10", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard20":{ "replicas":{ "core_node191":{ "core":"testNodeLost_shard20_replica_n191", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10110_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node200":{ "core":"testNodeLost_shard20_replica_n200", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10119_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node199":{ "core":"testNodeLost_shard20_replica_n199", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node198":{ "core":"testNodeLost_shard20_replica_n198", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node197":{ "core":"testNodeLost_shard20_replica_n197", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node196":{ "core":"testNodeLost_shard20_replica_n196", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node195":{ "core":"testNodeLost_shard20_replica_n195", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node194":{ "core":"testNodeLost_shard20_replica_n194", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node193":{ "core":"testNodeLost_shard20_replica_n193", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node192":{ "core":"testNodeLost_shard20_replica_n192", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10022_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"73330000-7fffffff", "state":"active"}, "shard10":{ "replicas":{ "core_node92":{ "core":"testNodeLost_shard10_replica_n92", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10022_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node91":{ "core":"testNodeLost_shard10_replica_n91", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10110_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node94":{ "core":"testNodeLost_shard10_replica_n94", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node93":{ "core":"testNodeLost_shard10_replica_n93", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node96":{ "core":"testNodeLost_shard10_replica_n96", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node95":{ "core":"testNodeLost_shard10_replica_n95", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node100":{ "core":"testNodeLost_shard10_replica_n100", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10119_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node98":{ "core":"testNodeLost_shard10_replica_n98", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node97":{ "core":"testNodeLost_shard10_replica_n97", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node99":{ "core":"testNodeLost_shard10_replica_n99", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"f3330000-ffffffff", "state":"active"}, "shard11":{ "replicas":{ "core_node108":{ "core":"testNodeLost_shard11_replica_n108", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node107":{ "core":"testNodeLost_shard11_replica_n107", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10114_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node106":{ "core":"testNodeLost_shard11_replica_n106", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node105":{ "core":"testNodeLost_shard11_replica_n105", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node104":{ "core":"testNodeLost_shard11_replica_n104", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node103":{ "core":"testNodeLost_shard11_replica_n103", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node102":{ "core":"testNodeLost_shard11_replica_n102", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node101":{ "core":"testNodeLost_shard11_replica_n101", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node110":{ "core":"testNodeLost_shard11_replica_n110", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node109":{ "core":"testNodeLost_shard11_replica_n109", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"0-ccbffff", "state":"active"}, "shard12":{ "replicas":{ "core_node119":{ "core":"testNodeLost_shard12_replica_n119", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10005_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node118":{ "core":"testNodeLost_shard12_replica_n118", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node117":{ "core":"testNodeLost_shard12_replica_n117", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node116":{ "core":"testNodeLost_shard12_replica_n116", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node115":{ "core":"testNodeLost_shard12_replica_n115", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10120_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node114":{ "core":"testNodeLost_shard12_replica_n114", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10127_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node113":{ "core":"testNodeLost_shard12_replica_n113", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10100_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node112":{ "core":"testNodeLost_shard12_replica_n112", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node111":{ "core":"testNodeLost_shard12_replica_n111", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node120":{ "core":"testNodeLost_shard12_replica_n120", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"ccc0000-1998ffff", "state":"active"}, "shard13":{ "replicas":{ "core_node129":{ "core":"testNodeLost_shard13_replica_n129", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10065_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node128":{ "core":"testNodeLost_shard13_replica_n128", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10121_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node127":{ "core":"testNodeLost_shard13_replica_n127", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node126":{ "core":"testNodeLost_shard13_replica_n126", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node125":{ "core":"testNodeLost_shard13_replica_n125", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node124":{ "core":"testNodeLost_shard13_replica_n124", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10123_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node123":{ "core":"testNodeLost_shard13_replica_n123", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10112_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node122":{ "core":"testNodeLost_shard13_replica_n122", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node121":{ "core":"testNodeLost_shard13_replica_n121", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node130":{ "core":"testNodeLost_shard13_replica_n130", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"19990000-2665ffff", "state":"active"}, "shard2":{ "replicas":{ "core_node18":{ "core":"testNodeLost_shard2_replica_n18", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node17":{ "core":"testNodeLost_shard2_replica_n17", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node19":{ "core":"testNodeLost_shard2_replica_n19", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10005_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node20":{ "core":"testNodeLost_shard2_replica_n20", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node12":{ "core":"testNodeLost_shard2_replica_n12", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node11":{ "core":"testNodeLost_shard2_replica_n11", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node14":{ "core":"testNodeLost_shard2_replica_n14", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10127_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node13":{ "core":"testNodeLost_shard2_replica_n13", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10100_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node16":{ "core":"testNodeLost_shard2_replica_n16", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node15":{ "core":"testNodeLost_shard2_replica_n15", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10120_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"8ccc0000-9998ffff", "state":"active"}, "shard3":{ "replicas":{ "core_node29":{ "core":"testNodeLost_shard3_replica_n29", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10065_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node28":{ "core":"testNodeLost_shard3_replica_n28", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10121_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node30":{ "core":"testNodeLost_shard3_replica_n30", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node21":{ "core":"testNodeLost_shard3_replica_n21", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node23":{ "core":"testNodeLost_shard3_replica_n23", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10112_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node22":{ "core":"testNodeLost_shard3_replica_n22", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node25":{ "core":"testNodeLost_shard3_replica_n25", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node24":{ "core":"testNodeLost_shard3_replica_n24", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10123_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node27":{ "core":"testNodeLost_shard3_replica_n27", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node26":{ "core":"testNodeLost_shard3_replica_n26", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"99990000-a665ffff", "state":"active"}, "shard14":{ "replicas":{ "core_node139":{ "core":"testNodeLost_shard14_replica_n139", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node138":{ "core":"testNodeLost_shard14_replica_n138", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node137":{ "core":"testNodeLost_shard14_replica_n137", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node136":{ "core":"testNodeLost_shard14_replica_n136", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10109_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node202":{ "node_name":"127.0.0.1:10072_solr", "core":"testNodeLost_shard14_replica_n201", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node135":{ "core":"testNodeLost_shard14_replica_n135", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10115_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node134":{ "core":"testNodeLost_shard14_replica_n134", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node133":{ "core":"testNodeLost_shard14_replica_n133", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10105_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node132":{ "core":"testNodeLost_shard14_replica_n132", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10117_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node140":{ "core":"testNodeLost_shard14_replica_n140", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"26660000-3332ffff", "state":"active"}, "shard15":{ "replicas":{ "core_node206":{ "node_name":"127.0.0.1:10059_solr", "core":"testNodeLost_shard15_replica_n205", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node149":{ "core":"testNodeLost_shard15_replica_n149", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10049_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node148":{ "core":"testNodeLost_shard15_replica_n148", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node146":{ "core":"testNodeLost_shard15_replica_n146", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node145":{ "core":"testNodeLost_shard15_replica_n145", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node144":{ "core":"testNodeLost_shard15_replica_n144", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node143":{ "core":"testNodeLost_shard15_replica_n143", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node142":{ "core":"testNodeLost_shard15_replica_n142", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node141":{ "core":"testNodeLost_shard15_replica_n141", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node150":{ "core":"testNodeLost_shard15_replica_n150", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"33330000-3fffffff", "state":"active"}, "shard4":{ "replicas":{ "core_node39":{ "core":"testNodeLost_shard4_replica_n39", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node204":{ "node_name":"127.0.0.1:10103_solr", "core":"testNodeLost_shard4_replica_n203", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node40":{ "core":"testNodeLost_shard4_replica_n40", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node32":{ "core":"testNodeLost_shard4_replica_n32", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10117_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node34":{ "core":"testNodeLost_shard4_replica_n34", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node33":{ "core":"testNodeLost_shard4_replica_n33", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10105_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node36":{ "core":"testNodeLost_shard4_replica_n36", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10109_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node35":{ "core":"testNodeLost_shard4_replica_n35", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10115_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node38":{ "core":"testNodeLost_shard4_replica_n38", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node37":{ "core":"testNodeLost_shard4_replica_n37", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"a6660000-b332ffff", "state":"active"}, "shard16":{ "replicas":{ "core_node159":{ "core":"testNodeLost_shard16_replica_n159", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node158":{ "core":"testNodeLost_shard16_replica_n158", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node157":{ "core":"testNodeLost_shard16_replica_n157", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node156":{ "core":"testNodeLost_shard16_replica_n156", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node155":{ "core":"testNodeLost_shard16_replica_n155", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10107_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node154":{ "core":"testNodeLost_shard16_replica_n154", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node153":{ "core":"testNodeLost_shard16_replica_n153", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node152":{ "core":"testNodeLost_shard16_replica_n152", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node151":{ "core":"testNodeLost_shard16_replica_n151", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node160":{ "core":"testNodeLost_shard16_replica_n160", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10122_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"40000000-4ccbffff", "state":"active"}, "shard5":{ "replicas":{ "core_node50":{ "core":"testNodeLost_shard5_replica_n50", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node41":{ "core":"testNodeLost_shard5_replica_n41", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node43":{ "core":"testNodeLost_shard5_replica_n43", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node42":{ "core":"testNodeLost_shard5_replica_n42", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node45":{ "core":"testNodeLost_shard5_replica_n45", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node44":{ "core":"testNodeLost_shard5_replica_n44", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node46":{ "core":"testNodeLost_shard5_replica_n46", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node49":{ "core":"testNodeLost_shard5_replica_n49", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10049_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node48":{ "core":"testNodeLost_shard5_replica_n48", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node208":{ "node_name":"127.0.0.1:10053_solr", "core":"testNodeLost_shard5_replica_n207", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"b3330000-bfffffff", "state":"active"}, "shard17":{ "replicas":{ "core_node169":{ "core":"testNodeLost_shard17_replica_n169", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node168":{ "core":"testNodeLost_shard17_replica_n168", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node167":{ "core":"testNodeLost_shard17_replica_n167", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node166":{ "core":"testNodeLost_shard17_replica_n166", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node165":{ "core":"testNodeLost_shard17_replica_n165", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node164":{ "core":"testNodeLost_shard17_replica_n164", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10125_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node163":{ "core":"testNodeLost_shard17_replica_n163", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10103_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node162":{ "core":"testNodeLost_shard17_replica_n162", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node161":{ "core":"testNodeLost_shard17_replica_n161", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10116_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node170":{ "core":"testNodeLost_shard17_replica_n170", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"4ccc0000-5998ffff", "state":"active"}, "shard18":{ "replicas":{ "core_node180":{ "core":"testNodeLost_shard18_replica_n180", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node179":{ "core":"testNodeLost_shard18_replica_n179", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node178":{ "core":"testNodeLost_shard18_replica_n178", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node177":{ "core":"testNodeLost_shard18_replica_n177", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node176":{ "core":"testNodeLost_shard18_replica_n176", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node175":{ "core":"testNodeLost_shard18_replica_n175", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node174":{ "core":"testNodeLost_shard18_replica_n174", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10101_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node173":{ "core":"testNodeLost_shard18_replica_n173", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node172":{ "core":"testNodeLost_shard18_replica_n172", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node171":{ "core":"testNodeLost_shard18_replica_n171", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"59990000-6665ffff", "state":"active"}, "shard19":{ "replicas":{ "core_node190":{ "core":"testNodeLost_shard19_replica_n190", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node189":{ "core":"testNodeLost_shard19_replica_n189", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10124_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node188":{ "core":"testNodeLost_shard19_replica_n188", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node187":{ "core":"testNodeLost_shard19_replica_n187", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node186":{ "core":"testNodeLost_shard19_replica_n186", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node185":{ "core":"testNodeLost_shard19_replica_n185", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10113_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node184":{ "core":"testNodeLost_shard19_replica_n184", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10106_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node183":{ "core":"testNodeLost_shard19_replica_n183", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10104_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node182":{ "core":"testNodeLost_shard19_replica_n182", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node181":{ "core":"testNodeLost_shard19_replica_n181", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"66660000-7332ffff", "state":"active"}, "shard1":{ "replicas":{ "core_node1":{ "core":"testNodeLost_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node6":{ "core":"testNodeLost_shard1_replica_n6", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node10":{ "core":"testNodeLost_shard1_replica_n10", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node7":{ "core":"testNodeLost_shard1_replica_n7", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10114_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node8":{ "core":"testNodeLost_shard1_replica_n8", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node9":{ "core":"testNodeLost_shard1_replica_n9", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{ "core":"testNodeLost_shard1_replica_n2", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node3":{ "core":"testNodeLost_shard1_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testNodeLost_shard1_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node5":{ "core":"testNodeLost_shard1_replica_n5", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"80000000-8ccbffff", "state":"active"}, "shard6":{ "replicas":{ "core_node60":{ "core":"testNodeLost_shard6_replica_n60", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10122_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node52":{ "core":"testNodeLost_shard6_replica_n52", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node51":{ "core":"testNodeLost_shard6_replica_n51", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node54":{ "core":"testNodeLost_shard6_replica_n54", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node53":{ "core":"testNodeLost_shard6_replica_n53", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node56":{ "core":"testNodeLost_shard6_replica_n56", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node55":{ "core":"testNodeLost_shard6_replica_n55", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10107_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node58":{ "core":"testNodeLost_shard6_replica_n58", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node57":{ "core":"testNodeLost_shard6_replica_n57", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node59":{ "core":"testNodeLost_shard6_replica_n59", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"c0000000-cccbffff", "state":"active"}, "shard7":{ "replicas":{ "core_node70":{ "core":"testNodeLost_shard7_replica_n70", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node61":{ "core":"testNodeLost_shard7_replica_n61", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10116_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node63":{ "core":"testNodeLost_shard7_replica_n63", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10103_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node62":{ "core":"testNodeLost_shard7_replica_n62", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node65":{ "core":"testNodeLost_shard7_replica_n65", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node64":{ "core":"testNodeLost_shard7_replica_n64", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10125_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node67":{ "core":"testNodeLost_shard7_replica_n67", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node66":{ "core":"testNodeLost_shard7_replica_n66", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node69":{ "core":"testNodeLost_shard7_replica_n69", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node68":{ "core":"testNodeLost_shard7_replica_n68", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"cccc0000-d998ffff", "state":"active"}, "shard8":{ "replicas":{ "core_node80":{ "core":"testNodeLost_shard8_replica_n80", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node72":{ "core":"testNodeLost_shard8_replica_n72", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node71":{ "core":"testNodeLost_shard8_replica_n71", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node74":{ "core":"testNodeLost_shard8_replica_n74", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10101_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node73":{ "core":"testNodeLost_shard8_replica_n73", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node76":{ "core":"testNodeLost_shard8_replica_n76", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node75":{ "core":"testNodeLost_shard8_replica_n75", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node78":{ "core":"testNodeLost_shard8_replica_n78", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node77":{ "core":"testNodeLost_shard8_replica_n77", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node79":{ "core":"testNodeLost_shard8_replica_n79", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"d9990000-e665ffff", "state":"active"}, "shard9":{ "replicas":{ "core_node90":{ "core":"testNodeLost_shard9_replica_n90", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node81":{ "core":"testNodeLost_shard9_replica_n81", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node83":{ "core":"testNodeLost_shard9_replica_n83", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10104_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node82":{ "core":"testNodeLost_shard9_replica_n82", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node85":{ "core":"testNodeLost_shard9_replica_n85", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10113_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node84":{ "core":"testNodeLost_shard9_replica_n84", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10106_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node87":{ "core":"testNodeLost_shard9_replica_n87", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node86":{ "core":"testNodeLost_shard9_replica_n86", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node89":{ "core":"testNodeLost_shard9_replica_n89", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10124_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node88":{ "core":"testNodeLost_shard9_replica_n88", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"e6660000-f332ffff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testNodeLost//clusterstate.json/463)={
"replicationFactor":"10",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"5",
"autoAddReplicas":"false",
"nrtReplicas":"10",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard20":{
"replicas":{
"core_node191":{
"core":"testNodeLost_shard20_replica_n191",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10110_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node200":{
"core":"testNodeLost_shard20_replica_n200",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10119_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node199":{
"core":"testNodeLost_shard20_replica_n199",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10020_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node198":{
"core":"testNodeLost_shard20_replica_n198",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10037_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node197":{
"core":"testNodeLost_shard20_replica_n197",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10038_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node196":{
"core":"testNodeLost_shard20_replica_n196",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.si
[...truncated too long message...]
dApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1137020262
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1137020262
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1137020262
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1304741452
[copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1304741452
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1304741452
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 78ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 340 minutes 23 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 75 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/75/
4 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:33429/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:38009/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:33429/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:38009/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([FB959E4A6DBE7BE5:51584DB8DA6DAE35]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33429/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 46 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([FB959E4A6DBE7BE5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([FB959E4A6DBE7BE5]:0)
FAILED: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic
Error Message:
one expected:<101> but was:<100>
Stack Trace:
java.lang.AssertionError: one expected:<101> but was:<100>
at __randomizedtesting.SeedInfo.seed([FB959E4A6DBE7BE5:506F835FB262FDCB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14213 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> 2983691 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/init-core-data-001
[junit4] 2> 2983693 WARN (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
[junit4] 2> 2983693 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2983694 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2983696 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001
[junit4] 2> 2983696 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2983696 INFO (Thread-4248) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2983696 INFO (Thread-4248) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2983705 ERROR (Thread-4248) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2983796 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:35017
[junit4] 2> 2983801 INFO (zkConnectionManagerCallback-5324-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2983812 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2983812 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2983813 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2983814 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2983814 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2983814 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2983814 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2983815 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2983815 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2983815 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2983816 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64ba8308{/solr,null,AVAILABLE}
[junit4] 2> 2983817 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2983817 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2983817 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2983818 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12b635aa{/solr,null,AVAILABLE}
[junit4] 2> 2983818 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6639a99{/solr,null,AVAILABLE}
[junit4] 2> 2983818 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@b00d0ed{SSL,[ssl, http/1.1]}{127.0.0.1:38009}
[junit4] 2> 2983818 INFO (jetty-launcher-5321-thread-2) [ ] o.e.j.s.Server Started @2983943ms
[junit4] 2> 2983818 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38009}
[junit4] 2> 2983819 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@66bc256{SSL,[ssl, http/1.1]}{127.0.0.1:53219}
[junit4] 2> 2983826 INFO (jetty-launcher-5321-thread-3) [ ] o.e.j.s.Server Started @2983950ms
[junit4] 2> 2983832 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53219}
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4b67866a{SSL,[ssl, http/1.1]}{127.0.0.1:33429}
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-4) [ ] o.e.j.s.Server Started @2983957ms
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33429}
[junit4] 2> 2983833 ERROR (jetty-launcher-5321-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2983833 ERROR (jetty-launcher-5321-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-10T06:38:10.613Z
[junit4] 2> 2983826 ERROR (jetty-launcher-5321-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2983836 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2983836 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2983836 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2983836 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2983836 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-10T06:38:10.616Z
[junit4] 2> 2983825 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2983837 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2983837 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2983833 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2983837 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2983837 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2983837 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-10T06:38:10.617Z
[junit4] 2> 2983845 INFO (zkConnectionManagerCallback-5328-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2983845 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16e9ea27{/solr,null,AVAILABLE}
[junit4] 2> 2983845 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48c07531{SSL,[ssl, http/1.1]}{127.0.0.1:57697}
[junit4] 2> 2983852 INFO (jetty-launcher-5321-thread-1) [ ] o.e.j.s.Server Started @2983976ms
[junit4] 2> 2983852 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57697}
[junit4] 2> 2983852 ERROR (jetty-launcher-5321-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2983852 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2983853 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2983853 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2983853 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2983853 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-10T06:38:10.633Z
[junit4] 2> 2983854 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2983859 INFO (zkConnectionManagerCallback-5326-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2983859 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2983860 INFO (zkConnectionManagerCallback-5332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2983866 INFO (zkConnectionManagerCallback-5330-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2983866 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2983872 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2984024 INFO (jetty-launcher-5321-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35017/solr
[junit4] 2> 2984044 INFO (zkConnectionManagerCallback-5336-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984045 DEBUG (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$45/1540906135@e819ee8
[junit4] 2> 2984066 INFO (zkConnectionManagerCallback-5338-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984189 INFO (jetty-launcher-5321-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35017/solr
[junit4] 2> 2984204 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984204 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984204 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984205 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984205 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984205 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 2984208 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2984208 DEBUG (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73949263645507590-127.0.0.1:57697_solr-n_0000000000
[junit4] 2> 2984208 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57697_solr
[junit4] 2> 2984209 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.Overseer Overseer (id=73949263645507590-127.0.0.1:57697_solr-n_0000000000) starting
[junit4] 2> 2984219 INFO (zkConnectionManagerCallback-5344-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984221 DEBUG (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$45/1540906135@27796c82
[junit4] 2> 2984260 INFO (zkConnectionManagerCallback-5346-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984266 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984266 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984267 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984267 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984267 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984268 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 2984269 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2984269 DEBUG (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73949263645507592-127.0.0.1:33429_solr-n_0000000001
[junit4] 2> 2984269 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73949263645507590-127.0.0.1:57697_solr-n_0000000000
[junit4] 2> 2984270 DEBUG (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73949263645507590-127.0.0.1:57697_solr-n_0000000000 to know if I could be the leader
[junit4] 2> 2984270 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33429_solr
[junit4] 2> 2984270 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984270 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984287 INFO (zkConnectionManagerCallback-5351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984288 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984288 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984288 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2984288 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984289 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984289 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984289 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2984294 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984294 INFO (zkCallback-5345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2984303 DEBUG (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 2984305 INFO (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2984306 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
[junit4] 2> 2984306 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984310 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 2984311 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 2984312 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984315 DEBUG (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:57697_solr as DOWN
[junit4] 2> 2984316 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2984316 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984317 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2984317 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57697_solr
[junit4] 2> 2984317 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984317 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984329 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984329 INFO (zkCallback-5337-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2984332 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984332 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984332 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
[junit4] 2> 2984332 INFO (zkCallback-5345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2984332 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2984332 DEBUG (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:57697_solr"} current state version: 0
[junit4] 2> 2984333 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2984333 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2984333 INFO (jetty-launcher-5321-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35017/solr
[junit4] 2> 2984340 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984340 INFO (zkCallback-5350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2984341 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 2984341 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:57697_solr, 127.0.0.1:33429_solr]
[junit4] 2> 2984344 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2984345 DEBUG (OverseerAutoScalingTriggerThread-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2984345 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 2984345 DEBUG (ScheduledTrigger-7730-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2984350 WARN (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2984392 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57697.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984403 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57697.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984403 INFO (zkConnectionManagerCallback-5359-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984403 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57697.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984403 INFO (zkConnectionManagerCallback-5362-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984405 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984405 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984405 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2984405 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984406 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984406 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984406 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2984408 INFO (jetty-launcher-5321-thread-1) [n:127.0.0.1:57697_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001/node1/.
[junit4] 2> 2984408 WARN (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2984423 DEBUG (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$45/1540906135@40d77299
[junit4] 2> 2984433 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984440 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33429.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984454 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984463 INFO (zkConnectionManagerCallback-5366-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984467 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33429.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984467 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33429.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984470 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984470 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984471 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2984471 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984471 INFO (jetty-launcher-5321-thread-4) [n:127.0.0.1:33429_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001/node4/.
[junit4] 2> 2984471 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984471 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984472 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 2984473 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2984474 DEBUG (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73949263645507596-127.0.0.1:38009_solr-n_0000000002
[junit4] 2> 2984474 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73949263645507592-127.0.0.1:33429_solr-n_0000000001
[junit4] 2> 2984474 DEBUG (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73949263645507592-127.0.0.1:33429_solr-n_0000000001 to know if I could be the leader
[junit4] 2> 2984474 DEBUG (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:38009_solr as DOWN
[junit4] 2> 2984475 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984476 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984476 DEBUG (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:38009_solr"} current state version: 0
[junit4] 2> 2984476 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2984476 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38009_solr
[junit4] 2> 2984477 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984477 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984477 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984477 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984477 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984478 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984478 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984478 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984478 INFO (zkCallback-5350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2984478 INFO (zkCallback-5345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2984479 INFO (zkCallback-5337-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2984488 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984489 INFO (zkCallback-5358-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2984493 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984493 INFO (zkCallback-5365-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2984541 INFO (zkConnectionManagerCallback-5373-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984541 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984542 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984542 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2984542 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984542 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984542 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984543 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2984543 WARN (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2984564 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38009.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984575 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38009.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984576 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38009.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984576 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984577 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984578 INFO (jetty-launcher-5321-thread-2) [n:127.0.0.1:38009_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001/node2/.
[junit4] 2> 2984654 INFO (jetty-launcher-5321-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35017/solr
[junit4] 2> 2984655 INFO (zkConnectionManagerCallback-5378-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984657 DEBUG (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$45/1540906135@1c8b5c1a
[junit4] 2> 2984658 INFO (zkConnectionManagerCallback-5380-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984666 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984667 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984667 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2984667 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984668 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984668 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984669 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 2984670 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2984671 DEBUG (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73949263645507599-127.0.0.1:53219_solr-n_0000000003
[junit4] 2> 2984671 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73949263645507596-127.0.0.1:38009_solr-n_0000000002
[junit4] 2> 2984671 DEBUG (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73949263645507596-127.0.0.1:38009_solr-n_0000000002 to know if I could be the leader
[junit4] 2> 2984672 DEBUG (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:53219_solr as DOWN
[junit4] 2> 2984673 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984673 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2984673 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53219_solr
[junit4] 2> 2984673 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984673 DEBUG (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:53219_solr"} current state version: 0
[junit4] 2> 2984674 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984674 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984674 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984675 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984675 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984675 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984675 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2984675 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984675 INFO (zkCallback-5365-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984676 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984676 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984676 INFO (zkCallback-5337-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984676 INFO (zkCallback-5358-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984678 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984678 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984678 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984678 INFO (zkCallback-5350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984678 INFO (zkCallback-5345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984678 INFO (zkCallback-5372-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984683 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984683 INFO (zkCallback-5379-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2984705 INFO (zkConnectionManagerCallback-5387-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984706 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984706 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984706 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2984706 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984707 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984707 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984708 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2984708 WARN (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2984733 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53219.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984747 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53219.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984748 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53219.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2984749 INFO (jetty-launcher-5321-thread-3) [n:127.0.0.1:53219_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001/node3/.
[junit4] 2> 2984774 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2984775 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2984815 INFO (zkConnectionManagerCallback-5390-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984819 INFO (zkConnectionManagerCallback-5395-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2984820 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2984820 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2984820 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2984820 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2984821 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2984821 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2984821 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2984897 INFO (qtp1781849107-20874) [n:127.0.0.1:57697_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2984899 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984899 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984900 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 2984900 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
[junit4] 2> 2984900 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984900 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984900 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{"operation":"overseerstatus"}
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 2984901 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 2984902 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 2984902 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
[junit4] 2> 2984914 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
[junit4] 2> 2984914 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : overseerstatus , {"operation":"overseerstatus"}
[junit4] 2> 2984918 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor Completed task:[/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 2984919 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/overseer/collection-queue-work/qnr-0000000000
[junit4] 2> 2984919 DEBUG (zkCallback-5337-thread-2) [ ] o.a.s.c.OverseerTaskQueue NodeDataChanged fired on path /overseer/collection-queue-work/qnr-0000000000 state SyncConnected latchEventType null
[junit4] 2> 2984919 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984919 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as completed.
[junit4] 2> 2984919 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984919 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 2984920 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 2984920 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2984920 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 2984920 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 2984920 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 2984920 DEBUG (OverseerThreadFactory-7732-thread-1) [ ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{leader=127.0.0.1:57697_solr,overseer_queue_size=0,overseer_work_queue_size=0,overseer_collection_queue_size=2,overseer_operations={am_i_leader={requests=5,errors=0,avgRequestsPerSecond=8.144921493455607,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.39515420000000007,medianRequestTime=0.302098,75thPcRequestTime=0.514162,95thPcRequestTime=0.601633,99thPcRequestTime=0.601633,999thPcRequestTime=0.601633},downnode={requests=3,errors=0,avgRequestsPerSecond=5.136289973482756,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.012300666666666666,medianRequestTime=0.012747,75thPcRequestTime=0.013263,95thPcRequestTime=0.013263,99thPcRequestTime=0.013263,999thPcRequestTime=0.013263}},collection_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=4.9435309033695,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.31234166666666663,medianRequestTime=0.315955,75thPcRequestTime=0.395488,95thPcRequestTime=0.395488,99thPcRequestTime=0.395488,999thPcRequestTime=0.395488},overseerstatus={requests=0,errors=0,avgRequestsPerSecond=0.0,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.0,medianRequestTime=0.0,75thPcRequestTime=0.0,95thPcRequestTime=0.0,99thPcRequestTime=0.0,999thPcRequestTime=0.0}},overseer_queue={},overseer_internal_queue={peek={avgRequestsPerSecond=1.6371783719293824,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.439052,medianRequestTime=0.439052,75thPcRequestTime=0.439052,95thPcRequestTime=0.439052,99thPcRequestTime=0.439052,999thPcRequestTime=0.439052}},collection_queue={peektopn_wait2000={avgRequestsPerSecond=1.652359876664488,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=589.46209,medianRequestTime=589.46209,75thPcRequestTime=589.46209,95thPcRequestTime=589.46209,99thPcRequestTime=589.46209,999thPcRequestTime=589.46209}}}
[junit4] 2> 2984921 INFO (qtp1781849107-20874) [n:127.0.0.1:57697_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=23
[junit4] 2> 2984923 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 2985109 WARN (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 2985122 WARN (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2985124 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2985141 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_45911_hdfs____.waa2ce/webapp
[junit4] 2> 2985349 DEBUG (ScheduledTrigger-7730-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2985595 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45911
[junit4] 2> 2985802 WARN (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2985804 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2985836 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51530_datanode____.femuar/webapp
[junit4] 2> 2986267 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51530
[junit4] 2> 2986357 DEBUG (ScheduledTrigger-7730-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2986533 WARN (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2986539 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2986579 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36392_datanode____yzsts/webapp
[junit4] 2> 2986684 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53803) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 2986712 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x668c3131315134: from storage DS-1a4b4b40-43b8-42ca-876d-68b2e001c6f3 node DatanodeRegistration(127.0.0.1:37981, datanodeUuid=a742c4f3-efbc-4cab-bcc3-994eeb26347e, infoPort=40601, infoSecurePort=0, ipcPort=49254, storageInfo=lv=-56;cid=testClusterID;nsid=1194327988;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2986712 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x668c3131315134: from storage DS-5ad84bb1-c2c0-43b8-a63b-948eabd91f7a node DatanodeRegistration(127.0.0.1:37981, datanodeUuid=a742c4f3-efbc-4cab-bcc3-994eeb26347e, infoPort=40601, infoSecurePort=0, ipcPort=49254, storageInfo=lv=-56;cid=testClusterID;nsid=1194327988;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 2986902 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 2986936 INFO (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 2986936 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 2987136 INFO (SUITE-MoveReplicaHDFSTest-seed#[FB959E4A6DBE7BE5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36392
[junit4] 2> 2987358 DEBUG (ScheduledTrigger-7730-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2987376 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:53803) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 2987404 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x668c315a48f949: from storage DS-dd817b4f-37fb-4e54-962d-8dd8999f327a node DatanodeRegistration(127.0.0.1:60662, datanodeUuid=54a53e55-8572-44ab-b67e-d8b5655c896b, infoPort=39319, infoSecurePort=0, ipcPort=44229, storageInfo=lv=-56;cid=testClusterID;nsid=1194327988;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2987404 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x668c315a48f949: from storage DS-b775105f-8631-47e3-b8d3-dd95ddecc82b node DatanodeRegistration(127.0.0.1:60662, datanodeUuid=54a53e55-8572-44ab-b67e-d8b5655c896b, infoPort=39319, infoSecurePort=0, ipcPort=44229, storageInfo=lv=-56;cid=testClusterID;nsid=1194327988;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 2987927 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNormalMove
[junit4] 2> 2987928 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2987928 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2987928 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2987929 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2987929 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2987929 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2987931 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2987936 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2987936 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2987936 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2987936 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30bc38f7{/solr,null,AVAILABLE}
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@37934c5d{SSL,[ssl, http/1.1]}{127.0.0.1:55165}
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.e.j.s.Server Started @2988061ms
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55165}
[junit4] 2> 2987937 ERROR (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2987937 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-10T06:38:14.717Z
[junit4] 2> 2987949 INFO (zkConnectionManagerCallback-5399-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2987949 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2988073 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35017/solr
[junit4] 2> 2988074 INFO (zkConnectionManagerCallback-5403-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2988076 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$45/1540906135@3bfccfb5
[junit4] 2> 2988077 INFO (zkConnectionManagerCallback-5405-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2988083 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2988083 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988085 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2988085 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2988086 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2988086 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2988087 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 2988088 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2988088 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73949263645507605-127.0.0.1:55165_solr-n_0000000004
[junit4] 2> 2988089 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73949263645507599-127.0.0.1:53219_solr-n_0000000003
[junit4] 2> 2988089 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73949263645507599-127.0.0.1:53219_solr-n_0000000003 to know if I could be the leader
[junit4] 2> 2988089 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:55165_solr as DOWN
[junit4] 2> 2988090 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2988090 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2988090 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55165_solr
[junit4] 2> 2988090 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2988091 DEBUG (OverseerStateUpdate-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:55165_solr"} current state version: 0
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988091 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988092 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988092 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988092 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 2988096 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988096 INFO (zkCallback-5358-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988096 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988096 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988096 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988096 INFO (zkCallback-5379-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 INFO (zkCallback-5365-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 INFO (zkCallback-5345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988097 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988097 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988097 INFO (zkCallback-5337-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 INFO (zkCallback-5394-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 INFO (zkCallback-5350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988097 INFO (zkCallback-5394-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988097 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988098 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988098 INFO (zkCallback-5372-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988098 INFO (zkCallback-5404-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988098 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988098 INFO (zkCallback-5386-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2988118 INFO (zkConnectionManagerCallback-5412-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2988119 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 2988120 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 2988120 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 2988120 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 2988120 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 2988120 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 2988121 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35017/solr ready
[junit4] 2> 2988121 WARN (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2988146 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55165.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2988159 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55165.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2988159 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55165.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43069555
[junit4] 2> 2988161 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [n:127.0.0.1:55165_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_FB959E4A6DBE7BE5-001/tempDir-001/node5/.
[junit4] 2> 2988191 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 2988192 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 2988236 INFO (zkConnectionManagerCallback-5415-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2988238 INFO (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[FB959E4A6DBE7BE5]) [ ] o.a.s.c.MoveReplicaTest total_jettys: 5
[junit4] 2> 2988268 INFO (qtp1585025282-20885) [n:127.0.0.1:33429_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=false&name=MoveReplicaHDFSTest_coll_false&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2988275 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2988275 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2988275 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000002
[junit4] 2> 2988275 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
[junit4] 2> 2988275 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 2988276 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 2988276 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
[junit4] 2> 2988276 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000002, ]
[junit4] 2> 2988276 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ed1]]
[junit4] 2> 2988276 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000002] as running
[junit4] 2> 2988276 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4] 2> "name":"MoveReplicaHDFSTest_coll_false",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000002]
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000002]
[junit4] 2> 2988277 DEBUG (OverseerCollectionConfigSetProcessor-73949263645507590-127.0.0.1:57697_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000002],[]
[junit4] 2> 2988277 DEBU
[...truncated too long message...]
e = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1340167127
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1340167127
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1340167127
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1810442434
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1810442434
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1810442434
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 40ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 286 minutes 14 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 74 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/74/
3 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove
Error Message:
Collection not found: MoveReplicaHDFSTest_coll_false
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: MoveReplicaHDFSTest_coll_false
at __randomizedtesting.SeedInfo.seed([EA2FB78042EDA802:4CF70027C70C6A18]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:853)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.cloud.MoveReplicaTest.addDocs(MoveReplicaTest.java:374)
at org.apache.solr.cloud.MoveReplicaTest.test(MoveReplicaTest.java:111)
at org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove(MoveReplicaHDFSTest.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10016_solr should have been deleted
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10016_solr should have been deleted
at __randomizedtesting.SeedInfo.seed([EA2FB78042EDA802:F2953F8C4CD865ED]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([EA2FB78042EDA802:B767A9098D2B0E4D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12576 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_EA2FB78042EDA802-001/init-core-data-001
[junit4] 2> 454375 INFO (SUITE-TestTriggerIntegration-seed#[EA2FB78042EDA802]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 454376 INFO (SUITE-TestTriggerIntegration-seed#[EA2FB78042EDA802]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 454377 INFO (SUITE-TestTriggerIntegration-seed#[EA2FB78042EDA802]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 454411 DEBUG (SUITE-TestTriggerIntegration-seed#[EA2FB78042EDA802]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10009_solr
[junit4] 2> 454411 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 454411 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 454414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 454415 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 454475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10009_solr]
[junit4] 2> 454475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 454475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 454476 DEBUG (ScheduledTrigger-1624-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 454487 WARN (SUITE-TestTriggerIntegration-seed#[EA2FB78042EDA802]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 454490 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 454490 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 454491 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 454491 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 454495 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 454508 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 454513 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 454514 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 454514 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 454527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr]
[junit4] 2> 454527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 454527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 454527 DEBUG (ScheduledTrigger-1629-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454548 DEBUG (ScheduledTrigger-1629-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454568 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454589 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454652 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454672 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454692 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454712 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454733 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454746 DEBUG (simCloudManagerPool-1628-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 454747 DEBUG (simCloudManagerPool-1628-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 454747 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 454748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 454748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 454753 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454772 DEBUG (simCloudManagerPool-1628-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 454772 DEBUG (simCloudManagerPool-1628-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 454773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 454773 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 454773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr]
[junit4] 2> 454774 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 454774 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 454774 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10010_solr
[junit4] 2> 454775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 454775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 454794 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 454794 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10009_solr
[junit4] 2> 454794 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 454794 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10009_solr
[junit4] 2> 454814 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 454814 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 454834 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 454834 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 454834 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10009_solr]
[junit4] 2> 454837 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"65fe5b78cbcd8cTalsd34ee26hypk3sj8jfryl2w",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":28708641469877644,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708641469877644],
[junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}}
[junit4] 2> 454837 DEBUG (ScheduledTrigger-1629-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 454935 DEBUG (AutoscalingActionExecutor-1630-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"65fe5b78cbcd8cTalsd34ee26hypk3sj8jfryl2w",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":28708641469877644,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708641469877644],
[junit4] 2> "_enqueue_time_":28708643655608244,
[junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}}
[junit4] 2> 454958 DEBUG (AutoscalingActionExecutor-1630-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 454958 DEBUG (AutoscalingActionExecutor-1630-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 454958 DEBUG (AutoscalingActionExecutor-1630-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1158 ms for event id=65fe5b78cbcd8cTalsd34ee26hypk3sj8jfryl2w
[junit4] 2> 454959 DEBUG (simCloudManagerPool-1628-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 454959 DEBUG (simCloudManagerPool-1628-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 454960 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 454960 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 454960 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 455001 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
[junit4] 2> 455005 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 1
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10009_solr
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 1
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 455009 INFO (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 455009 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 455009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 455009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 455009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 455009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 455009 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10009_solr
[junit4] 2> 455010 INFO (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 455010 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 455010 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 455010 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 455011 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 455011 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 455011 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 455011 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 455251 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 455251 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 455259 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 455259 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 455266 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 455279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 455279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 455279 DEBUG (ScheduledTrigger-1632-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455303 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455323 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455344 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455364 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455365 DEBUG (simCloudManagerPool-1631-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 455365 DEBUG (simCloudManagerPool-1631-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 455373 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 455374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 455374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 455384 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455395 DEBUG (simCloudManagerPool-1631-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 455395 DEBUG (simCloudManagerPool-1631-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 455396 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 455396 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 455397 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 455397 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction}], enabled=true}
[junit4] 2> 455397 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 455398 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455398 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 455404 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455418 DEBUG (simCloudManagerPool-1631-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 455418 DEBUG (simCloudManagerPool-1631-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 455418 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455419 DEBUG (ScheduledTrigger-1632-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 455419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 455419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 455422 DEBUG (simCloudManagerPool-1631-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 455422 DEBUG (simCloudManagerPool-1631-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 455424 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 455430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 455439 DEBUG (ScheduledTrigger-1632-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455439 DEBUG (ScheduledTrigger-1632-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 455439 DEBUG (ScheduledTrigger-1632-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10012_solr at time 28708673725769394
[junit4] 2> 455445 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 455459 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455459 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 455465 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 455480 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455480 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 455485 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 455526 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455526 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 455526 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10012_solr] added at times [28708673725769394], now=28708678082960094
[junit4] 2> 455527 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"65fe62fb65a6b2Talsd34ee26hypk3sj8jfryl2z",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708673725769394,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708673725769394],
[junit4] 2> "nodeNames":["127.0.0.1:10012_solr"]}}
[junit4] 2> 455527 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 455591 DEBUG (ScheduledTrigger-1632-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10012_solr
[junit4] 2> 455592 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"65fe62fb65a6b2Talsd34ee26hypk3sj8jfryl2z",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708673725769394,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708673725769394],
[junit4] 2> "_enqueue_time_":28708678135965294,
[junit4] 2> "nodeNames":["127.0.0.1:10012_solr"]}}
[junit4] 2> 455603 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 455603 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 455603 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 598 ms for event id=65fe62fb65a6b2Talsd34ee26hypk3sj8jfryl2z
[junit4] 2> 455704 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 455704 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455704 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 455704 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10013_solr at time 28708686969176794
[junit4] 2> 455724 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 455724 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455724 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 455744 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 455745 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455745 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 455764 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 455765 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 455765 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 455765 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10013_solr] added at times [28708686969176794], now=28708690011969294
[junit4] 2> 455765 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"65fe6610c402daTalsd34ee26hypk3sj8jfryl32",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708686969176794,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708686969176794],
[junit4] 2> "nodeNames":["127.0.0.1:10013_solr"]}}
[junit4] 2> 455765 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 455892 DEBUG (ScheduledTrigger-1632-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10013_solr
[junit4] 2> 455892 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"65fe6610c402daTalsd34ee26hypk3sj8jfryl32",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708686969176794,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708686969176794],
[junit4] 2> "_enqueue_time_":28708690032917844,
[junit4] 2> "nodeNames":["127.0.0.1:10013_solr"]}}
[junit4] 2> 455898 WARN (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: test1 for trigger event: {
[junit4] 2> "id":"65fe6610c402daTalsd34ee26hypk3sj8jfryl32",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708686969176794,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708686969176794],
[junit4] 2> "_enqueue_time_":28708690032917844,
[junit4] 2> "nodeNames":["127.0.0.1:10013_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction.process(TestTriggerIntegration.java:911) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
[junit4] 2> ... 6 more
[junit4] 2> 455899 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 455899 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 455899 DEBUG (AutoscalingActionExecutor-1633-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 359 ms for event id=65fe6610c402daTalsd34ee26hypk3sj8jfryl32
[junit4] 2> 455933 INFO (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 455933 INFO (TEST-TestTriggerIntegration.testListeners-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 4
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 4
[junit4] 2> ## - update 4
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - FAILED 1
[junit4] 2> ## - STARTED 2
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 455946 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 455946 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 455946 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 455946 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 455946 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 455946 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 455947 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10011_solr
[junit4] 2> 455947 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10012_solr
[junit4] 2> 455947 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 455947 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 455947 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 455948 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 455948 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 455948 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 455948 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 455948 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 455963 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 455963 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 455963 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion -1
[junit4] 2> 455963 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 455975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10013_solr]
[junit4] 2> 455978 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 455978 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 455978 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 455999 DEBUG (ScheduledTrigger-1635-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456019 DEBUG (ScheduledTrigger-1635-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456039 DEBUG (ScheduledTrigger-1635-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456059 DEBUG (ScheduledTrigger-1635-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456079 DEBUG (simCloudManagerPool-1634-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456079 DEBUG (simCloudManagerPool-1634-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 456080 DEBUG (ScheduledTrigger-1635-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456081 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 456081 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456081 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 456098 DEBUG (simCloudManagerPool-1634-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456107 DEBUG (ScheduledTrigger-1635-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456108 DEBUG (simCloudManagerPool-1634-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 456109 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 456109 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 456109 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10013_solr]
[junit4] 2> 456109 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 456109 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 456109 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456109 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 456127 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 456130 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456130 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 456130 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10014_solr at time 28708708256458744
[junit4] 2> 456147 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 456150 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456150 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 456207 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 456207 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456207 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 456207 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10014_solr] added at times [28708708256458744], now=28708712137760494
[junit4] 2> 456208 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"65fe6b059627f8Talsd34ee26hypk3sj8jfryl35",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708708256458744,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708708256458744],
[junit4] 2> "nodeNames":["127.0.0.1:10014_solr"]}}
[junit4] 2> 456211 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 456267 DEBUG (ScheduledTrigger-1635-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10014_solr
[junit4] 2> 456267 DEBUG (AutoscalingActionExecutor-1636-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"65fe6b059627f8Talsd34ee26hypk3sj8jfryl35",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708708256458744,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708708256458744],
[junit4] 2> "_enqueue_time_":28708712329140594,
[junit4] 2> "nodeNames":["127.0.0.1:10014_solr"]}}
[junit4] 2> 456311 DEBUG (AutoscalingActionExecutor-1636-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 456311 DEBUG (AutoscalingActionExecutor-1636-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 456311 DEBUG (AutoscalingActionExecutor-1636-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2216 ms for event id=65fe6b059627f8Talsd34ee26hypk3sj8jfryl35
[junit4] 2> 456312 DEBUG (simCloudManagerPool-1634-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456312 DEBUG (simCloudManagerPool-1634-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 456318 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 456319 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 456361 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 456362 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 456366 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 456366 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 456366 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10014_solr
[junit4] 2> 456366 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 456366 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 456366 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 456367 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 456367 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 456367 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 456367 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 456367 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 456383 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 456384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 456421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion -1
[junit4] 2> 456421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 456439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10013_solr]
[junit4] 2> 456448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 456449 DEBUG (ScheduledTrigger-1638-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456494 DEBUG (ScheduledTrigger-1638-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456515 DEBUG (ScheduledTrigger-1638-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456521 DEBUG (simCloudManagerPool-1637-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456522 DEBUG (simCloudManagerPool-1637-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 456527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 456528 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 1
[junit4] 2> 456528 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 456530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456530 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 456531 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10013_solr
[junit4] 2> 456547 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 456548 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion -1
[junit4] 2> 456548 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 456552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10015_solr]
[junit4] 2> 456552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10010_solr
[junit4] 2> 456558 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456558 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
[junit4] 2> 456558 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456558 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
[junit4] 2> 456578 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456599 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456619 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456640 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456647 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD TRIGGERS
[junit4] 2> 456660 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456668 INFO (simCloudManagerPool-1640-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 456668 DEBUG (simCloudManagerPool-1640-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456668 DEBUG (simCloudManagerPool-1640-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 456671 INFO (simCloudManagerPool-1640-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 456671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 456672 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10015_solr]
[junit4] 2> 456672 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 456672 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 456672 DEBUG (ScheduledTrigger-1641-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456672 DEBUG (ScheduledTrigger-1641-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 456680 DEBUG (ScheduledTrigger-1641-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456689 INFO (simCloudManagerPool-1640-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 456690 DEBUG (simCloudManagerPool-1640-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 456690 DEBUG (simCloudManagerPool-1640-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 21
[junit4] 2> 456691 INFO (simCloudManagerPool-1640-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 456691 INFO (simCloudManagerPool-1640-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 456691 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 21
[junit4] 2> 456691 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10013_solr, 127.0.0.1:10015_solr]
[junit4] 2> 456691 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
[junit4] 2> 456691 DEBUG (ScheduledTrigger-1641-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 456700 DEBUG (ScheduledTrigger-1641-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 456700 DEBUG (ScheduledTrigger-1641-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456700 DEBUG (ScheduledTrigger-1641-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 456701 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD NODE 1
[junit4] 2> 456712 DEBUG (ScheduledTrigger-1641-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 456720 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 456721 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456721 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 456721 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10016_solr at time 28708737808235594
[junit4] 2> 456732 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 456741 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 456741 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 456741 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 456741 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10016_solr] added at times [28708737808235594], now=28708738826808194
[junit4] 2> 456741 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"65fe71e7027c4aTalsd34ee26hypk3sj8jfryl38",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708737808235594,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708737808235594],
[junit4] 2> "nodeNames":["127.0.0.1:10016_solr"]}}
[junit4] 2> 456742 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger]
[junit4] 2> 456803 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
[junit4] 2> 456804 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10010_solr
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeMarkersRegistration -Dtests.seed=EA2FB78042EDA802 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=hr-HR -Dtests.timezone=Pacific/Noumea -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.44s J1 | TestTriggerIntegration.testNodeMarkersRegistration <<<
[junit4] > Throwable #1: java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10016_solr should have been deleted
[junit4] > at __randomizedtesting.SeedInfo.seed([EA2FB78042EDA802:F2953F8C4CD865ED]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:863)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 456819 DEBUG (ScheduledTrigger-1641-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10016_solr
[junit4] 2> 456827 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 456827 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 22
[junit4] 2> 456828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 22
[junit4] 2> 456828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 456828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 22, lastZnodeVersion 22
[junit4] 2> 456828 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10010_solr
[junit4] 2> 456828 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10015_solr
[junit4] 2> 456828 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 456828 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 456831 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 456831 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 456831 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 456832 DEBUG (AutoscalingActionExecutor-1642-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"65fe71e7027c4aTalsd34ee26hypk3sj8jfryl38",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":28708737808235594,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[28708737808235594],
[junit4] 2> "_enqueue_time_":28708738856584544,
[junit4] 2> "nodeNames":["127.0.0.1:10016_solr"]}}
[junit4] 2> 456832 WARN (AutoscalingActionExecutor-1642-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 456832 DEBUG (AutoscalingActionExecutor-1642-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 35 ms for event id=65fe71e7027c4aTalsd34ee26hypk3sj8jfryl38
[junit4] 2> 456839 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 456839 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 456839 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[EA2FB78042EDA802]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 456845 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 456845 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 23
[junit4] 2> 456846 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 23, lastZnodeVersion -1
[junit4] 2> 456846 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 23
[junit4] 2> 456859 WARN (MetricsHistoryHandler-1627-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 456867 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10013_solr]
[junit4] 2> 456997 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 456997 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 23, lastZnodeVersion 23
[junit4] 2> 456997 DEBUG (ScheduledTrigger-1644-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457003 DEBUG (simCloudManagerPool-1643-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 457003 DEBUG (simCloudManagerPool-1643-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 24
[junit4] 2> 457020 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 24
[junit4] 2> 457027 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 457027 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 24, lastZnodeVersion 24
[junit4] 2> 457028 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457049 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457069 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457089 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457101 DEBUG (simCloudManagerPool-1643-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 457101 DEBUG (simCloudManagerPool-1643-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 25
[junit4] 2> 457110 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457110 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 25
[junit4] 2> 457110 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 457110 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 25, lastZnodeVersion 25
[junit4] 2> 457130 DEBUG (ScheduledTrigger-1644-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457150 DEBUG (ScheduledTrigger-1644-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 457159 DEBUG (simCloudManagerPool-1643-thread-7) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 457160 DEBUG (simCloudManagerPool-1643-thread-7) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 26
[junit4] 2> 457161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 26
[junit4] 2> 457161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 457161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 26, lastZnodeVersion 26
[...truncated too long message...]
file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2011564224
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2011564224
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2011564224
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null725530840
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null725530840
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null725530840
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 101ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 172 minutes 0 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 73 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/73/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/128)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard1_1_0":{ "state":"active", "replicas":{ "core_node14":{ "core":"testSplitIntegration_collection_shard1_1_0_replica1", "SEARCHER.searcher.maxDoc":6, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "base_url":"http://127.0.0.1:10004/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":6}, "core_node13":{ "core":"testSplitIntegration_collection_shard1_1_0_replica0", "SEARCHER.searcher.maxDoc":6, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":6}}}, "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}}, "range":"0-7fffffff", "state":"active"}, "shard1_1_1":{ "state":"active", "replicas":{"core_node15":{ "core":"testSplitIntegration_collection_shard1_1_1_replica0", "SEARCHER.searcher.maxDoc":6, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":6}}}, "shard1":{ "stateTimestamp":"1528484561188054500", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528484561220512400", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":12, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "base_url":"http://127.0.0.1:10004/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":12}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":12, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":12}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528484561220214550", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":13, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":13}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":13, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "base_url":"http://127.0.0.1:10004/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":13}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/128)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard1_1_0":{
"state":"active",
"replicas":{
"core_node14":{
"core":"testSplitIntegration_collection_shard1_1_0_replica1",
"SEARCHER.searcher.maxDoc":6,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":6},
"core_node13":{
"core":"testSplitIntegration_collection_shard1_1_0_replica0",
"SEARCHER.searcher.maxDoc":6,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":6}}},
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25}},
"range":"0-7fffffff",
"state":"active"},
"shard1_1_1":{
"state":"active",
"replicas":{"core_node15":{
"core":"testSplitIntegration_collection_shard1_1_1_replica0",
"SEARCHER.searcher.maxDoc":6,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":6}}},
"shard1":{
"stateTimestamp":"1528484561188054500",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528484561220512400",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":12,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":12},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":12,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":12}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528484561220214550",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":13,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":13},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":13,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":13}}}}}
at __randomizedtesting.SeedInfo.seed([A8BF159B93290B83:9131ACDBBCD6C27D]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode
Error Message:
trigger did not fire
Stack Trace:
java.lang.AssertionError: trigger did not fire
at __randomizedtesting.SeedInfo.seed([A8BF159B93290B83:F5008385C64849B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:300)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13258 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> 1271151 INFO (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_A8BF159B93290B83-001/init-core-data-001
[junit4] 2> 1271152 WARN (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
[junit4] 2> 1271152 INFO (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1271187 INFO (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1271291 DEBUG (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 1271292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1271292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1271292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1271292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1271298 WARN (SUITE-TestLargeCluster-seed#[A8BF159B93290B83]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 1271380 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1271409 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1271409 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1271444 DEBUG (ScheduledTrigger-3988-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271448 INFO (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 1271449 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1271450 INFO (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1271452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1271454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1271455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1271455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1271463 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1271475 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1271475 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1271475 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1271475 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1271475 WARN (ScheduledTrigger-3988-thread-3) [ ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
[junit4] 2> 1271475 ERROR (ScheduledTrigger-3988-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
[junit4] 2> java.lang.RuntimeException: Trigger has been closed
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:588) [java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1271476 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1271476 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1271493 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1271493 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1271493 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1271493 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1271495 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1271496 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1271496 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1271496 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271517 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271537 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271558 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271578 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271722 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271742 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271748 DEBUG (simCloudManagerPool-3992-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1271752 DEBUG (simCloudManagerPool-3992-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1271752 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1271753 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1271753 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1271762 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271782 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271803 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271823 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271843 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271864 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271884 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271906 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271926 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271947 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271967 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1271987 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272007 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272028 DEBUG (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272129 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272150 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272170 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272222 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272242 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272263 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272283 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272304 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272324 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272344 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272371 DEBUG (simCloudManagerPool-3992-thread-25) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard2: {"core_node18":{
[junit4] 2> "core":"testSearchRate_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testSearchRate"}}
[junit4] 2> 1272373 DEBUG (simCloudManagerPool-3992-thread-26) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard1: {"core_node1":{
[junit4] 2> "core":"testSearchRate_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10091_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testSearchRate"}}
[junit4] 2> 1272374 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272394 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272411 INFO (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 1862 ms
[junit4] 2> 1272414 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272434 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272456 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272476 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272496 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272517 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272537 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272558 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272578 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272599 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272619 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272721 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272742 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272762 DEBUG (ScheduledTrigger-3993-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272773 DEBUG (simCloudManagerPool-3992-thread-27) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1272774 DEBUG (simCloudManagerPool-3992-thread-27) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1272774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1272775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1272775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1272782 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272815 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272835 DEBUG (simCloudManagerPool-3992-thread-28) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1272835 DEBUG (simCloudManagerPool-3992-thread-28) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1272835 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272842 WARN (ScheduledTrigger-3993-thread-4) [ ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
[junit4] 2> 1272854 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1272854 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1272855 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1272856 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272876 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272896 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272916 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272937 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272957 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272978 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1272998 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273018 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273040 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273060 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273080 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273101 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273154 DEBUG (ScheduledTrigger-3993-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273157 WARN (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
[junit4] 2> 1273174 DEBUG (ScheduledTrigger-3993-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1273193 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"658df30c079899Tdb6iobn45kztzgf9ywnh9evzg",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":28585047666170009,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10051_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10079_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10038_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10073_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10050_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10097_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10037_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "hotCollections":{"testSearchRate":40.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10073_solr":4.0,
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10050_solr":4.0,
[junit4] 2> "127.0.0.1:10037_solr":4.0,
[junit4] 2> "127.0.0.1:10079_solr":4.0,
[junit4] 2> "127.0.0.1:10038_solr":4.0,
[junit4] 2> "127.0.0.1:10051_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10097_solr":4.0},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 1273194 DEBUG (ScheduledTrigger-3993-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 1273256 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"658df30c079899Tdb6iobn45kztzgf9ywnh9evzg",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":28585047666170009,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10051_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10079_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10038_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10073_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10050_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10097_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10037_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "hotCollections":{"testSearchRate":40.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10073_solr":4.0,
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10050_solr":4.0,
[junit4] 2> "127.0.0.1:10037_solr":4.0,
[junit4] 2> "127.0.0.1:10079_solr":4.0,
[junit4] 2> "127.0.0.1:10038_solr":4.0,
[junit4] 2> "127.0.0.1:10051_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10097_solr":4.0},
[junit4] 2> "_enqueue_time_":28585065236642309,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 1273257 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"658df30c079899Tdb6iobn45kztzgf9ywnh9evzg",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":28585047666170009,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10051_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10079_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10038_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10073_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10050_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10097_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10037_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "hotCollections":{"testSearchRate":40.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10073_solr":4.0,
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10050_solr":4.0,
[junit4] 2> "127.0.0.1:10037_solr":4.0,
[junit4] 2> "127.0.0.1:10079_solr":4.0,
[junit4] 2> "127.0.0.1:10038_solr":4.0,
[junit4] 2> "127.0.0.1:10051_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10097_solr":4.0},
[junit4] 2> "_enqueue_time_":28585065236642309,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 1273455 INFO (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10072_solr&type=NRT
[junit4] 2> 1273556 INFO (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> 1273672 INFO (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10094_solr&type=NRT
[junit4] 2> 1273672 INFO (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28585089126 sessionWrapper.createTime 28585070046097909, this.sessionWrapper.createTime 28585070046097909
[junit4] 2> 1273672 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"658df30c079899Tdb6iobn45kztzgf9ywnh9evzg",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":28585047666170009,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10051_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10079_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10038_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10073_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10050_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10097_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10037_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "__start__":3,
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "hotCollections":{"testSearchRate":40.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10073_solr":4.0,
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10050_solr":4.0,
[junit4] 2> "127.0.0.1:10037_solr":4.0,
[junit4] 2> "127.0.0.1:10079_solr":4.0,
[junit4] 2> "127.0.0.1:10038_solr":4.0,
[junit4] 2> "127.0.0.1:10051_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10097_solr":4.0},
[junit4] 2> "_enqueue_time_":28585065236642309,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@690a35ba, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@4973cc19, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@58293981], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 1273684 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10072_solr&type=NRT
[junit4] 2> 1273697 INFO (simCloudManagerPool-3992-thread-30) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10072_solr for creating new replica
[junit4] 2> 1274129 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> 1274146 INFO (simCloudManagerPool-3992-thread-39) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10056_solr for creating new replica
[junit4] 2> 1274192 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10094_solr&type=NRT
[junit4] 2> 1274200 INFO (simCloudManagerPool-3992-thread-45) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10094_solr for creating new replica
[junit4] 2> 1274253 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1274253 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: search_rate_trigger after 100ms
[junit4] 2> 1274253 DEBUG (AutoscalingActionExecutor-3994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 51555 ms for event id=658df30c079899Tdb6iobn45kztzgf9ywnh9evzg
[junit4] 2> 1274287 INFO (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 1274294 INFO (TEST-TestLargeCluster.testSearchRate-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 100
[junit4] 2> ## Empty nodes: 77
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system, testSearchRate]
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 24
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ## * testSearchRate 23
[junit4] 2> ## - active 23
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - ADDREPLICA 3
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETESTATUS 3
[junit4] 2> ## - REQUESTSTATUS 3
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 7
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: search_rate_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 1274321 INFO (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
[junit4] 2> 1274321 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1274322 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1274322 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1274322 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1274322 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1274322 INFO (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1274323 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1274323 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1274323 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1274324 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1274324 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1274324 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1274324 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[A8BF159B93290B83]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1274336 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1274336 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1274337 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 1274337 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1274360 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1274390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1274390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1274394 DEBUG (ScheduledTrigger-3996-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274494 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274508 DEBUG (simCloudManagerPool-3995-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1274508 DEBUG (simCloudManagerPool-3995-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1274514 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1274520 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1274520 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1274534 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274581 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274602 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274610 DEBUG (simCloudManagerPool-3995-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1274610 DEBUG (simCloudManagerPool-3995-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1274610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1274611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1274611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=start, class=org.apache.solr.cloud.autoscaling.sim.TestLargeCluster$StartTriggerAction}, {name=compute, class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestLargeCluster$FinishTriggerAction}], enabled=true}
[junit4] 2> 1274611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1274611 DEBUG (ScheduledTrigger-3996-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1274611 DEBUG (ScheduledTrigger-3996-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 1274622 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274632 DEBUG (ScheduledTrigger-3996-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1274632 DEBUG (ScheduledTrigger-3996-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 1274696 DEBUG (ScheduledTrigger-3996-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1274696 DEBUG (ScheduledTrigger-3996-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1274696 DEBUG (ScheduledTrigger-3996-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 1275464 DEBUG (ScheduledTrigger-3996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live no
[...truncated too long message...]
file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null119688266
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null119688266
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null119688266
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1590989765
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1590989765
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1590989765
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 62ms :: artifacts dl 8ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 176 minutes 21 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 72 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/72/
10 tests failed.
FAILED: org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
Error Message:
MockDirectoryWrapper: cannot close: there are still 34 open files: {_1.tvd=1, _1_MockRandom_0.pos=1, _1.nvd=1, _3_Lucene70_0.dvd=1, _0.nvd=1, _2.fdt=1, _3_MockRandom_0.doc=1, _4.fdx=1, _2_MockRandom_0.tio=1, _2.nvd=1, _2.tvd=1, _0_MockRandom_0.doc=1, _0_Lucene70_0.dvd=1, _2_MockRandom_0.pos=1, _1.fdt=1, _5.fdx=1, _1_MockRandom_0.doc=1, _3.nvd=1, _5.fdt=1, _5.tvx=1, _1_Lucene70_0.dvd=1, _4.tvd=1, _3.tvd=1, _3_MockRandom_0.tim=1, _0.fdt=1, _0_MockRandom_0.pos=1, _0.tvd=1, _2_MockRandom_0.doc=1, _4.fdt=1, _4.tvx=1, _2_Lucene70_0.dvd=1, _3_MockRandom_0.pos=1, _5.tvd=1, _3.fdt=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 34 open files: {_1.tvd=1, _1_MockRandom_0.pos=1, _1.nvd=1, _3_Lucene70_0.dvd=1, _0.nvd=1, _2.fdt=1, _3_MockRandom_0.doc=1, _4.fdx=1, _2_MockRandom_0.tio=1, _2.nvd=1, _2.tvd=1, _0_MockRandom_0.doc=1, _0_Lucene70_0.dvd=1, _2_MockRandom_0.pos=1, _1.fdt=1, _5.fdx=1, _1_MockRandom_0.doc=1, _3.nvd=1, _5.fdt=1, _5.tvx=1, _1_Lucene70_0.dvd=1, _4.tvd=1, _3.tvd=1, _3_MockRandom_0.tim=1, _0.fdt=1, _0_MockRandom_0.pos=1, _0.tvd=1, _2_MockRandom_0.doc=1, _4.fdt=1, _4.tvx=1, _2_Lucene70_0.dvd=1, _3_MockRandom_0.pos=1, _5.tvd=1, _3.fdt=1}
at __randomizedtesting.SeedInfo.seed([AD843BE13B0BB289:FAD21C009EBAF7D7]:0)
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
at org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:341)
at org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce(TestIndexWriterWithThreads.java:464)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _3.fdt
at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:150)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:82)
at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:172)
at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:705)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4364)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4052)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
FAILED: org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState
Error Message:
expected:<27> but was:<28>
Stack Trace:
java.lang.AssertionError: expected:<27> but was:<28>
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:C251F60F3E20E8A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState(TestLeaderInitiatedRecoveryThread.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test
Error Message:
Trigger was not fired
Stack Trace:
java.lang.AssertionError: Trigger was not fired
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:14786B2B34C1CCA6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test(SystemLogListenerTest.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:C1644A7855FB0711]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:1411DD8EA0FD40F3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState(TestTriggerIntegration.java:381)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:9C1AE031B3300534]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState(TestTriggerIntegration.java:693)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10013_solr should have been deleted
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10013_solr should have been deleted
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:8496DCFD94086CB1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.ResourceLoaderTest.testAwareCompatibility
Error Message:
Configuration Error: missing parameter 'minGramSize'
Stack Trace:
java.lang.IllegalArgumentException: Configuration Error: missing parameter 'minGramSize'
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:DE4C1681E311480B]:0)
at org.apache.lucene.analysis.util.AbstractAnalysisFactory.require(AbstractAnalysisFactory.java:97)
at org.apache.lucene.analysis.util.AbstractAnalysisFactory.requireInt(AbstractAnalysisFactory.java:157)
at org.apache.lucene.analysis.ngram.NGramFilterFactory.<init>(NGramFilterFactory.java:44)
at org.apache.solr.core.ResourceLoaderTest.testAwareCompatibility(ResourceLoaderTest.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic
Error Message:
{} expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: {} expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:37D649E445E12770]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode
Error Message:
trigger did not fire
Stack Trace:
java.lang.AssertionError: trigger did not fire
at __randomizedtesting.SeedInfo.seed([9C2C54F19A3DA15E:3BC3495255702E46]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:300)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 1723 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterWithThreads -Dtests.method=testIOExceptionDuringAbortWithThreadsOnlyOnce -Dtests.seed=AD843BE13B0BB289 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ar-SY -Dtests.timezone=Etc/GMT+12 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.70s J0 | TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce <<<
[junit4] > Throwable #1: java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 34 open files: {_1.tvd=1, _1_MockRandom_0.pos=1, _1.nvd=1, _3_Lucene70_0.dvd=1, _0.nvd=1, _2.fdt=1, _3_MockRandom_0.doc=1, _4.fdx=1, _2_MockRandom_0.tio=1, _2.nvd=1, _2.tvd=1, _0_MockRandom_0.doc=1, _0_Lucene70_0.dvd=1, _2_MockRandom_0.pos=1, _1.fdt=1, _5.fdx=1, _1_MockRandom_0.doc=1, _3.nvd=1, _5.fdt=1, _5.tvx=1, _1_Lucene70_0.dvd=1, _4.tvd=1, _3.tvd=1, _3_MockRandom_0.tim=1, _0.fdt=1, _0_MockRandom_0.pos=1, _0.tvd=1, _2_MockRandom_0.doc=1, _4.fdt=1, _4.tvx=1, _2_Lucene70_0.dvd=1, _3_MockRandom_0.pos=1, _5.tvd=1, _3.fdt=1}
[junit4] > at __randomizedtesting.SeedInfo.seed([AD843BE13B0BB289:FAD21C009EBAF7D7]:0)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
[junit4] > at org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:341)
[junit4] > at org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce(TestIndexWriterWithThreads.java:464)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.RuntimeException: unclosed IndexInput: _3.fdt
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
[junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:150)
[junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
[junit4] > at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsReader(AssertingStoredFieldsFormat.java:43)
[junit4] > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:82)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:172)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:705)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4364)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4052)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {date=PostingsFormat(name=MockRandom), field=PostingsFormat(name=MockRandom), docid=BlockTreeOrds(blocksize=128), titleTokenized=Lucene50(blocksize=128), id=BlockTreeOrds(blocksize=128), body=PostingsFormat(name=MockRandom), title=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{dv=DocValuesFormat(name=Lucene70), docid_intDV=DocValuesFormat(name=Direct), field=DocValuesFormat(name=Lucene70), titleDV=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=703, maxMBSortInHeap=6.872576717663296, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@27a95463), locale=ar-SY, timezone=Etc/GMT+12
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=179514880,total=469237760
[junit4] 2> NOTE: All tests run in this JVM: [TestTermVectorsWriter, TestSimpleExplanations, TestSloppyPhraseQuery, TestCachingCollector, TestFilterIterator, TestMultiThreadTermVectors, TestFlex, TestPendingSoftDeletes, TestLucene70DocValuesFormat, TestMultiPhraseQuery, TestDeterminizeLexicon, TestVirtualMethod, TestSimpleSearchEquivalence, TestTermVectors, TestSpanCollection, TestSoftDeletesDirectoryReaderWrapper, TestNRTCachingDirectory, TestDirectoryReaderReopen, TestGeoEncodingUtils, TestDocIdsWriter, TestStressDeletes, TestIndexableField, TestLiveFieldValues, TestStandardAnalyzer, TestAllFilesHaveChecksumFooter, TestBlockPostingsFormat, TestForUtil, TestIndexWriterOutOfFileDescriptors, TestDocValuesQueries, TestIndexWriter, TestIndexWriterFromReader, TestConstantScoreQuery, TestNoDeletionPolicy, TestBasicModelIne, TestDocValuesIndexing, TestIndexWriterNRTIsCurrent, TestMaxTermFrequency, LimitedFiniteStringsIteratorTest, TestDuelingCodecsAtNight, TestFilterDirectoryReader, TestPolygon2D, TestSortedSetSelector, TestSort, TestAxiomaticF3LOG, TestFlushByRamOrCountsPolicy, TestWordlistLoader, TestQueryBuilder, TestBagOfPositions, TestBasicModelG, TestNotDocIdSet, TestPriorityQueue, TestOneMergeWrappingMergePolicy, TestMultiPhraseEnum, TestDocsAndPositions, TestCodecs, TestIndexOptions, TestMSBRadixSorter, TestDoubleValuesSource, TestBytesStore, TestBoolean2, TestTryDelete, TestTwoPhaseCommitTool, TestTopFieldCollector, TestFilterWeight, TestDocumentWriter, TestFilterLeafReader, TestPagedBytes, TestSpanMultiTermQueryWrapper, TestFieldInvertState, TestIntsRef, TestIndexWriterExceptions, TestUniqueTermCount, TestPhrasePrefixQuery, TestBoostQuery, TestWANDScorer, TestPositionIncrement, TestSearchAfter, Test2BPagedBytes, TestIndexWriterWithThreads]
[junit4] Completed [444/495 (1!)] on J0 in 24.41s, 13 tests, 1 error <<< FAILURES!
[...truncated 10661 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.SystemLogListenerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SystemLogListenerTest_9C2C54F19A3DA15E-001/init-core-data-001
[junit4] 2> 22254 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 22692 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.e.j.u.log Logging initialized @22901ms to org.eclipse.jetty.util.log.Slf4jLog
[junit4] 2> 22764 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 24943 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 25018 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SystemLogListenerTest_9C2C54F19A3DA15E-001/tempDir-001
[junit4] 2> 25143 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 25191 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 25226 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 26259 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41760
[junit4] 2> 27989 ERROR (Thread-1) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 28537 INFO (zkConnectionManagerCallback-4-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 31019 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 31054 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 31072 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 31310 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 31310 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 31327 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 31347 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 31347 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 31347 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 31457 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 31457 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 31458 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 31576 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51ed4600{/solr,null,AVAILABLE}
[junit4] 2> 31577 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@783c7062{/solr,null,AVAILABLE}
[junit4] 2> 31639 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44cb4b11{/solr,null,AVAILABLE}
[junit4] 2> 32134 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1118f89{SSL,[ssl, http/1.1]}{127.0.0.1:35284}
[junit4] 2> 32151 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@78f7fc48{SSL,[ssl, http/1.1]}{127.0.0.1:39792}
[junit4] 2> 32151 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1e5e9f26{SSL,[ssl, http/1.1]}{127.0.0.1:38461}
[junit4] 2> 32152 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server Started @32378ms
[junit4] 2> 32153 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server Started @32379ms
[junit4] 2> 32153 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39792}
[junit4] 2> 32153 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38461}
[junit4] 2> 32168 INFO (jetty-launcher-1-thread-3) [ ] o.e.j.s.Server Started @32394ms
[junit4] 2> 32168 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35284}
[junit4] 2> 32561 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
[junit4] 2> 32779 ERROR (jetty-launcher-1-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 32779 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 32780 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 32797 ERROR (jetty-launcher-1-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 32798 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 32798 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 32798 ERROR (jetty-launcher-1-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 32798 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 32798 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 32814 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 32814 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 32815 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-05T14:37:48.754Z
[junit4] 2> 32834 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 32834 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 32834 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-05T14:37:48.773Z
[junit4] 2> 32835 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 32835 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 32835 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-05T14:37:48.774Z
[junit4] 2> 32984 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 33360 INFO (zkConnectionManagerCallback-6-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 33546 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 33570 INFO (zkConnectionManagerCallback-10-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 33571 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 33702 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 33704 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 34528 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10274a45fd10001, likely client has closed socket
[junit4] 2> 41247 INFO (jetty-launcher-1-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41760/solr
[junit4] 2> 41257 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41760/solr
[junit4] 2> 41258 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41760/solr
[junit4] 2> 41660 INFO (zkConnectionManagerCallback-18-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 41661 INFO (zkConnectionManagerCallback-20-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 41680 INFO (zkConnectionManagerCallback-22-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 42327 INFO (zkConnectionManagerCallback-24-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 42364 INFO (zkConnectionManagerCallback-26-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 42384 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 48081 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 48098 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 48101 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38461_solr
[junit4] 2> 48115 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 48175 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39792_solr
[junit4] 2> 48188 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35284_solr
[junit4] 2> 48190 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.Overseer Overseer (id=72748793322340361-127.0.0.1:38461_solr-n_0000000000) starting
[junit4] 2> 48664 INFO (zkCallback-23-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 48796 INFO (zkCallback-27-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 48797 INFO (zkCallback-25-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 49684 INFO (zkConnectionManagerCallback-39-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 49830 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 49924 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41760/solr ready
[junit4] 2> 52014 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 52015 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38461_solr
[junit4] 2> 52117 INFO (zkConnectionManagerCallback-45-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52255 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 52274 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41760/solr ready
[junit4] 2> 52310 INFO (zkCallback-25-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 52506 INFO (zkCallback-23-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 52558 INFO (zkConnectionManagerCallback-51-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52578 INFO (zkCallback-27-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 52751 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 52753 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41760/solr ready
[junit4] 2> 52761 INFO (zkCallback-38-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 53446 WARN (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 53530 WARN (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 53693 WARN (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 53969 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 54106 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 56418 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 56418 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 56489 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38461.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56519 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38461.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56520 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38461.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56540 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39792.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56625 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35284.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56880 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:38461_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SystemLogListenerTest_9C2C54F19A3DA15E-001/tempDir-001/node2/.
[junit4] 2> 56930 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:38461_solr, 127.0.0.1:39792_solr, 127.0.0.1:35284_solr]
[junit4] 2> 56942 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39792.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56942 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39792.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56944 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35284.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56944 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35284.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@483b2858
[junit4] 2> 56949 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 56950 DEBUG (OverseerAutoScalingTriggerThread-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 56959 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:39792_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SystemLogListenerTest_9C2C54F19A3DA15E-001/tempDir-001/node1/.
[junit4] 2> 56990 DEBUG (ScheduledTrigger-19-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 56995 INFO (jetty-launcher-1-thread-3) [n:127.0.0.1:35284_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SystemLogListenerTest_9C2C54F19A3DA15E-001/tempDir-001/node3/.
[junit4] 2> 58052 DEBUG (ScheduledTrigger-19-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 59056 DEBUG (ScheduledTrigger-19-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 59800 INFO (zkConnectionManagerCallback-54-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 60020 INFO (zkConnectionManagerCallback-59-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 60064 DEBUG (ScheduledTrigger-19-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 60077 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 60150 INFO (SUITE-SystemLogListenerTest-seed#[9C2C54F19A3DA15E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41760/solr ready
[junit4] 2> 61078 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 62087 DEBUG (ScheduledTrigger-19-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 63107 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 64828 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 65834 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 66843 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 67358 INFO (qtp1828561993-30) [n:127.0.0.1:38461_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=.system&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 67603 INFO (OverseerThreadFactory-21-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection .system
[junit4] 2> 67851 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 68163 INFO (OverseerStateUpdate-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35284/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 68253 INFO (OverseerStateUpdate-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38461/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 68267 INFO (OverseerStateUpdate-72748793322340361-127.0.0.1:38461_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39792/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 68855 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 69548 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr x:.system_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n3&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 69547 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr x:.system_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 69601 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr x:.system_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 69602 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr x:.system_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 69622 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr x:.system_shard1_replica_n3] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 69876 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 70880 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 70999 WARN (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.Config You should not use LATEST as luceneMatchVersion property: if you use this setting, and then Solr upgrades to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an actual Lucene version.
[junit4] 2> 71233 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 71269 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 71541 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 71840 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 71866 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 71867 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 71883 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 72255 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema [.system_shard1_replica_n1] Schema name=_system collection or core
[junit4] 2> 72345 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema [.system_shard1_replica_n2] Schema name=_system collection or core
[junit4] 2> 72526 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.s.IndexSchema [.system_shard1_replica_n3] Schema name=_system collection or core
[junit4] 2> 72891 DEBUG (ScheduledTrigger-19-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 73374 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 73407 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 73446 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 73533 INFO (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/.system/managed-schema
[junit4] 2> 73550 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/.system/managed-schema already exists - no need to create it
[junit4] 2> 73553 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/.system/schema.xml to /configs/.system/schema.xml.bak
[junit4] 2> 73554 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/.system/managed-schema already exists - no need to create it
[junit4] 2> 73554 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/.system/schema.xml no longer exists.
[junit4] 2> 73586 INFO (qtp1828561993-32) [n:127.0.0.1:38461_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n2' using configuration from collection .system, trusted=true
[junit4] 2> 73590 INFO (qtp427109654-38) [n:127.0.0.1:35284_solr c:.system s:shard1 r:core_node4 x:.system_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n1' using configuration from collection .system, trusted=true
[junit4] 2> 73570 WARN (qtp1255470152-47) [n:127.0.0.1:39792_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_n3] o.a.s.s.ManagedIndexSchemaFactory Error persisting managed schema resource managed-schema
[junit4] 2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /configs/.system/schema.xml
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:876) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$delete$1(SolrZkClient.java:245) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:244) ~[java/:?]
[junit4] 2> at org.apache.solr.schema.ManagedIndexSchemaFactory.zkUgradeToManagedSchema(ManagedIndexSchemaFactory.java:349) ~[java/:?]
[junit4] 2> at org.apache.solr.schema.ManagedIndexSchemaFactory.upgradeToManagedSchema(ManagedIndexSchemaFactory.java:268) ~[java/:?]
[junit4] 2> at org.apache.solr.schema.ManagedIndexSchemaFactory.create(ManagedIndexSchemaFactory.java:186) ~[java/:?]
[junit4] 2> at org.apache.solr.schema.ManagedIndexSchemaFactory.create(ManagedIndexSchemaFactory.java:45) ~[java/:?]
[junit4] 2> at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:70) ~[java/:?]
[junit4] 2> at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:119) ~[java/:?]
[junit4] 2> at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:92) ~[java/:?]
[junit4] 2> at org.apache.solr.core.CoreContainer.getConfigSet(CoreContainer.java:1153) ~[java/:?]
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1100) ~[java/:?]
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1014) ~[java/:?]
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:94) ~[java/:?]
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:380) [java/:?]
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395) [java/:?]
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) [java/:?]
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) [java/:?]
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734) [java/:?]
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715) [java/:?]
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) [java/:?]
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) [java/:?]
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) [java/:?]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) [java/:?]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) [jetty-servlet-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) [jetty-servlet-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:531) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [jetty-server-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [jetty-io-9.4.10.v20180503.jar:9.4.10.v20180503]
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [jetty-io-9.4.10.v201805
[...truncated too long message...]
BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null202976728
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null202976728
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null202976728
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2077696285
[copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2077696285
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2077696285
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 57ms :: artifacts dl 14ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 339 minutes 52 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)