You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/05/21 01:05:08 UTC
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 60 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/60/
5 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([9B07576659325420:F8CC61E4C0FD270D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode
Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=4},status={state=notfound,msg=Did not find [search_rate_trigger3/605211f350745eT5xkx7cwhbsfs634ryvu6258lt/0] in any tasks queue}}
Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=4},status={state=notfound,msg=Did not find [search_rate_trigger3/605211f350745eT5xkx7cwhbsfs634ryvu6258lt/0] in any tasks queue}}
at __randomizedtesting.SeedInfo.seed([9B07576659325420:B99599E46EF8DB5D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:668)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([9B07576659325420:C64F49EF96F4F26F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:579)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
last state: DocCollection(testNodeLost//clusterstate.json/106)={ "replicationFactor":"10", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"5", "autoAddReplicas":"false", "nrtReplicas":"10", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard20":{ "replicas":{ "core_node191":{ "core":"testNodeLost_shard20_replica_n191", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node200":{ "core":"testNodeLost_shard20_replica_n200", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node199":{ "core":"testNodeLost_shard20_replica_n199", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node198":{ "core":"testNodeLost_shard20_replica_n198", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node242":{ "node_name":"127.0.0.1:10068_solr", "core":"testNodeLost_shard20_replica_n241", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node197":{ "core":"testNodeLost_shard20_replica_n197", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node196":{ "core":"testNodeLost_shard20_replica_n196", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node195":{ "core":"testNodeLost_shard20_replica_n195", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node194":{ "core":"testNodeLost_shard20_replica_n194", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node192":{ "core":"testNodeLost_shard20_replica_n192", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"73330000-7fffffff", "state":"active"}, "shard10":{ "replicas":{ "core_node92":{ "core":"testNodeLost_shard10_replica_n92", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node91":{ "core":"testNodeLost_shard10_replica_n91", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node94":{ "core":"testNodeLost_shard10_replica_n94", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node96":{ "core":"testNodeLost_shard10_replica_n96", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node95":{ "core":"testNodeLost_shard10_replica_n95", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node244":{ "node_name":"127.0.0.1:10046_solr", "core":"testNodeLost_shard10_replica_n243", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node100":{ "core":"testNodeLost_shard10_replica_n100", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node98":{ "core":"testNodeLost_shard10_replica_n98", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node97":{ "core":"testNodeLost_shard10_replica_n97", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node99":{ "core":"testNodeLost_shard10_replica_n99", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"f3330000-ffffffff", "state":"active"}, "shard11":{ "replicas":{ "core_node108":{ "core":"testNodeLost_shard11_replica_n108", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node107":{ "core":"testNodeLost_shard11_replica_n107", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node105":{ "core":"testNodeLost_shard11_replica_n105", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node104":{ "core":"testNodeLost_shard11_replica_n104", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node268":{ "node_name":"127.0.0.1:10044_solr", "core":"testNodeLost_shard11_replica_n267", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node224":{ "node_name":"127.0.0.1:10078_solr", "core":"testNodeLost_shard11_replica_n223", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node256":{ "node_name":"127.0.0.1:10000_solr", "core":"testNodeLost_shard11_replica_n255", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node102":{ "core":"testNodeLost_shard11_replica_n102", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node110":{ "core":"testNodeLost_shard11_replica_n110", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node240":{ "node_name":"127.0.0.1:10063_solr", "core":"testNodeLost_shard11_replica_n239", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"0-ccbffff", "state":"active"}, "shard12":{ "replicas":{ "core_node119":{ "core":"testNodeLost_shard12_replica_n119", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node118":{ "core":"testNodeLost_shard12_replica_n118", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node117":{ "core":"testNodeLost_shard12_replica_n117", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node116":{ "core":"testNodeLost_shard12_replica_n116", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node115":{ "core":"testNodeLost_shard12_replica_n115", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node114":{ "core":"testNodeLost_shard12_replica_n114", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node113":{ "core":"testNodeLost_shard12_replica_n113", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node112":{ "core":"testNodeLost_shard12_replica_n112", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node111":{ "core":"testNodeLost_shard12_replica_n111", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node120":{ "core":"testNodeLost_shard12_replica_n120", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"ccc0000-1998ffff", "state":"active"}, "shard13":{ "replicas":{ "core_node126":{ "core":"testNodeLost_shard13_replica_n126", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node125":{ "core":"testNodeLost_shard13_replica_n125", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node246":{ "node_name":"127.0.0.1:10085_solr", "core":"testNodeLost_shard13_replica_n245", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node124":{ "core":"testNodeLost_shard13_replica_n124", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node123":{ "core":"testNodeLost_shard13_replica_n123", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node122":{ "core":"testNodeLost_shard13_replica_n122", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node121":{ "core":"testNodeLost_shard13_replica_n121", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node274":{ "node_name":"127.0.0.1:10027_solr", "core":"testNodeLost_shard13_replica_n273", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node130":{ "core":"testNodeLost_shard13_replica_n130", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node250":{ "node_name":"127.0.0.1:10011_solr", "core":"testNodeLost_shard13_replica_n249", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"19990000-2665ffff", "state":"active"}, "shard2":{ "replicas":{ "core_node18":{ "core":"testNodeLost_shard2_replica_n18", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node17":{ "core":"testNodeLost_shard2_replica_n17", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node19":{ "core":"testNodeLost_shard2_replica_n19", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node20":{ "core":"testNodeLost_shard2_replica_n20", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node12":{ "core":"testNodeLost_shard2_replica_n12", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node11":{ "core":"testNodeLost_shard2_replica_n11", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node14":{ "core":"testNodeLost_shard2_replica_n14", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node13":{ "core":"testNodeLost_shard2_replica_n13", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node16":{ "core":"testNodeLost_shard2_replica_n16", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node15":{ "core":"testNodeLost_shard2_replica_n15", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"8ccc0000-9998ffff", "state":"active"}, "shard3":{ "replicas":{ "core_node248":{ "node_name":"127.0.0.1:10040_solr", "core":"testNodeLost_shard3_replica_n247", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node30":{ "core":"testNodeLost_shard3_replica_n30", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node276":{ "node_name":"127.0.0.1:10088_solr", "core":"testNodeLost_shard3_replica_n275", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node21":{ "core":"testNodeLost_shard3_replica_n21", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node252":{ "node_name":"127.0.0.1:10014_solr", "core":"testNodeLost_shard3_replica_n251", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node23":{ "core":"testNodeLost_shard3_replica_n23", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node22":{ "core":"testNodeLost_shard3_replica_n22", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node25":{ "core":"testNodeLost_shard3_replica_n25", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node24":{ "core":"testNodeLost_shard3_replica_n24", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node26":{ "core":"testNodeLost_shard3_replica_n26", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"99990000-a665ffff", "state":"active"}, "shard14":{ "replicas":{ "core_node218":{ "node_name":"127.0.0.1:10031_solr", "core":"testNodeLost_shard14_replica_n217", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node139":{ "core":"testNodeLost_shard14_replica_n139", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node138":{ "core":"testNodeLost_shard14_replica_n138", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node137":{ "core":"testNodeLost_shard14_replica_n137", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node136":{ "core":"testNodeLost_shard14_replica_n136", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node135":{ "core":"testNodeLost_shard14_replica_n135", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node134":{ "core":"testNodeLost_shard14_replica_n134", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node133":{ "core":"testNodeLost_shard14_replica_n133", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node132":{ "core":"testNodeLost_shard14_replica_n132", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node131":{ "core":"testNodeLost_shard14_replica_n131", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"26660000-3332ffff", "state":"active"}, "shard4":{ "replicas":{ "core_node39":{ "core":"testNodeLost_shard4_replica_n39", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node32":{ "core":"testNodeLost_shard4_replica_n32", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node31":{ "core":"testNodeLost_shard4_replica_n31", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node220":{ "node_name":"127.0.0.1:10097_solr", "core":"testNodeLost_shard4_replica_n219", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node34":{ "core":"testNodeLost_shard4_replica_n34", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node33":{ "core":"testNodeLost_shard4_replica_n33", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node36":{ "core":"testNodeLost_shard4_replica_n36", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node35":{ "core":"testNodeLost_shard4_replica_n35", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node38":{ "core":"testNodeLost_shard4_replica_n38", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node37":{ "core":"testNodeLost_shard4_replica_n37", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"a6660000-b332ffff", "state":"active"}, "shard15":{ "replicas":{ "core_node149":{ "core":"testNodeLost_shard15_replica_n149", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node148":{ "core":"testNodeLost_shard15_replica_n148", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node147":{ "core":"testNodeLost_shard15_replica_n147", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node202":{ "node_name":"127.0.0.1:10072_solr", "core":"testNodeLost_shard15_replica_n201", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node146":{ "core":"testNodeLost_shard15_replica_n146", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node234":{ "node_name":"127.0.0.1:10062_solr", "core":"testNodeLost_shard15_replica_n233", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node145":{ "core":"testNodeLost_shard15_replica_n145", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node143":{ "core":"testNodeLost_shard15_replica_n143", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node141":{ "core":"testNodeLost_shard15_replica_n141", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node150":{ "core":"testNodeLost_shard15_replica_n150", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"33330000-3fffffff", "state":"active"}, "shard16":{ "replicas":{ "core_node159":{ "core":"testNodeLost_shard16_replica_n159", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node158":{ "core":"testNodeLost_shard16_replica_n158", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node157":{ "core":"testNodeLost_shard16_replica_n157", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node156":{ "core":"testNodeLost_shard16_replica_n156", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node155":{ "core":"testNodeLost_shard16_replica_n155", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node154":{ "core":"testNodeLost_shard16_replica_n154", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node153":{ "core":"testNodeLost_shard16_replica_n153", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node152":{ "core":"testNodeLost_shard16_replica_n152", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node151":{ "core":"testNodeLost_shard16_replica_n151", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node160":{ "core":"testNodeLost_shard16_replica_n160", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"40000000-4ccbffff", "state":"active"}, "shard5":{ "replicas":{ "core_node204":{ "node_name":"127.0.0.1:10056_solr", "core":"testNodeLost_shard5_replica_n203", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node50":{ "core":"testNodeLost_shard5_replica_n50", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node236":{ "node_name":"127.0.0.1:10069_solr", "core":"testNodeLost_shard5_replica_n235", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node41":{ "core":"testNodeLost_shard5_replica_n41", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node43":{ "core":"testNodeLost_shard5_replica_n43", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node45":{ "core":"testNodeLost_shard5_replica_n45", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node47":{ "core":"testNodeLost_shard5_replica_n47", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node46":{ "core":"testNodeLost_shard5_replica_n46", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node49":{ "core":"testNodeLost_shard5_replica_n49", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node48":{ "core":"testNodeLost_shard5_replica_n48", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"b3330000-bfffffff", "state":"active"}, "shard17":{ "replicas":{ "core_node206":{ "node_name":"127.0.0.1:10059_solr", "core":"testNodeLost_shard17_replica_n205", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node226":{ "node_name":"127.0.0.1:10075_solr", "core":"testNodeLost_shard17_replica_n225", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node258":{ "node_name":"127.0.0.1:10009_solr", "core":"testNodeLost_shard17_replica_n257", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node168":{ "core":"testNodeLost_shard17_replica_n168", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node166":{ "core":"testNodeLost_shard17_replica_n166", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node165":{ "core":"testNodeLost_shard17_replica_n165", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node164":{ "core":"testNodeLost_shard17_replica_n164", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node230":{ "node_name":"127.0.0.1:10024_solr", "core":"testNodeLost_shard17_replica_n229", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node163":{ "core":"testNodeLost_shard17_replica_n163", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node170":{ "core":"testNodeLost_shard17_replica_n170", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"4ccc0000-5998ffff", "state":"active"}, "shard18":{ "replicas":{ "core_node180":{ "core":"testNodeLost_shard18_replica_n180", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node214":{ "node_name":"127.0.0.1:10091_solr", "core":"testNodeLost_shard18_replica_n213", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node178":{ "core":"testNodeLost_shard18_replica_n178", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node177":{ "core":"testNodeLost_shard18_replica_n177", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node210":{ "node_name":"127.0.0.1:10037_solr", "core":"testNodeLost_shard18_replica_n209", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node176":{ "core":"testNodeLost_shard18_replica_n176", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node175":{ "core":"testNodeLost_shard18_replica_n175", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node174":{ "core":"testNodeLost_shard18_replica_n174", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node173":{ "core":"testNodeLost_shard18_replica_n173", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node171":{ "core":"testNodeLost_shard18_replica_n171", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"59990000-6665ffff", "state":"active"}, "shard19":{ "replicas":{ "core_node190":{ "core":"testNodeLost_shard19_replica_n190", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node189":{ "core":"testNodeLost_shard19_replica_n189", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node188":{ "core":"testNodeLost_shard19_replica_n188", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node187":{ "core":"testNodeLost_shard19_replica_n187", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node186":{ "core":"testNodeLost_shard19_replica_n186", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node185":{ "core":"testNodeLost_shard19_replica_n185", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node262":{ "node_name":"127.0.0.1:10041_solr", "core":"testNodeLost_shard19_replica_n261", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node182":{ "core":"testNodeLost_shard19_replica_n182", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node270":{ "node_name":"127.0.0.1:10025_solr", "core":"testNodeLost_shard19_replica_n269", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node181":{ "core":"testNodeLost_shard19_replica_n181", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"66660000-7332ffff", "state":"active"}, "shard1":{ "replicas":{ "core_node238":{ "node_name":"127.0.0.1:10015_solr", "core":"testNodeLost_shard1_replica_n237", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node266":{ "node_name":"127.0.0.1:10028_solr", "core":"testNodeLost_shard1_replica_n265", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node222":{ "node_name":"127.0.0.1:10002_solr", "core":"testNodeLost_shard1_replica_n221", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node254":{ "node_name":"127.0.0.1:10030_solr", "core":"testNodeLost_shard1_replica_n253", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node10":{ "core":"testNodeLost_shard1_replica_n10", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node7":{ "core":"testNodeLost_shard1_replica_n7", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node8":{ "core":"testNodeLost_shard1_replica_n8", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{ "core":"testNodeLost_shard1_replica_n2", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testNodeLost_shard1_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node5":{ "core":"testNodeLost_shard1_replica_n5", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"80000000-8ccbffff", "state":"active"}, "shard6":{ "replicas":{ "core_node60":{ "core":"testNodeLost_shard6_replica_n60", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node52":{ "core":"testNodeLost_shard6_replica_n52", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node51":{ "core":"testNodeLost_shard6_replica_n51", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node54":{ "core":"testNodeLost_shard6_replica_n54", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node53":{ "core":"testNodeLost_shard6_replica_n53", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node56":{ "core":"testNodeLost_shard6_replica_n56", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node55":{ "core":"testNodeLost_shard6_replica_n55", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node58":{ "core":"testNodeLost_shard6_replica_n58", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node57":{ "core":"testNodeLost_shard6_replica_n57", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node59":{ "core":"testNodeLost_shard6_replica_n59", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"c0000000-cccbffff", "state":"active"}, "shard7":{ "replicas":{ "core_node228":{ "node_name":"127.0.0.1:10050_solr", "core":"testNodeLost_shard7_replica_n227", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node70":{ "core":"testNodeLost_shard7_replica_n70", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node63":{ "core":"testNodeLost_shard7_replica_n63", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node65":{ "core":"testNodeLost_shard7_replica_n65", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node232":{ "node_name":"127.0.0.1:10008_solr", "core":"testNodeLost_shard7_replica_n231", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node64":{ "core":"testNodeLost_shard7_replica_n64", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node66":{ "core":"testNodeLost_shard7_replica_n66", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node260":{ "node_name":"127.0.0.1:10087_solr", "core":"testNodeLost_shard7_replica_n259", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node68":{ "core":"testNodeLost_shard7_replica_n68", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node208":{ "node_name":"127.0.0.1:10053_solr", "core":"testNodeLost_shard7_replica_n207", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"cccc0000-d998ffff", "state":"active"}, "shard8":{ "replicas":{ "core_node216":{ "node_name":"127.0.0.1:10034_solr", "core":"testNodeLost_shard8_replica_n215", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node80":{ "core":"testNodeLost_shard8_replica_n80", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node71":{ "core":"testNodeLost_shard8_replica_n71", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node74":{ "core":"testNodeLost_shard8_replica_n74", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node212":{ "node_name":"127.0.0.1:10094_solr", "core":"testNodeLost_shard8_replica_n211", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node73":{ "core":"testNodeLost_shard8_replica_n73", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node76":{ "core":"testNodeLost_shard8_replica_n76", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node75":{ "core":"testNodeLost_shard8_replica_n75", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node78":{ "core":"testNodeLost_shard8_replica_n78", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node77":{ "core":"testNodeLost_shard8_replica_n77", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"d9990000-e665ffff", "state":"active"}, "shard9":{ "replicas":{ "core_node90":{ "core":"testNodeLost_shard9_replica_n90", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node81":{ "core":"testNodeLost_shard9_replica_n81", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node82":{ "core":"testNodeLost_shard9_replica_n82", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node85":{ "core":"testNodeLost_shard9_replica_n85", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node87":{ "core":"testNodeLost_shard9_replica_n87", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node264":{ "node_name":"127.0.0.1:10092_solr", "core":"testNodeLost_shard9_replica_n263", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node86":{ "core":"testNodeLost_shard9_replica_n86", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node89":{ "core":"testNodeLost_shard9_replica_n89", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node88":{ "core":"testNodeLost_shard9_replica_n88", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node272":{ "node_name":"127.0.0.1:10039_solr", "core":"testNodeLost_shard9_replica_n271", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"e6660000-f332ffff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testNodeLost//clusterstate.json/106)={
"replicationFactor":"10",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"5",
"autoAddReplicas":"false",
"nrtReplicas":"10",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard20":{
"replicas":{
"core_node191":{
"core":"testNodeLost_shard20_replica_n191",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10064_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node200":{
"core":"testNodeLost_shard20_replica_n200",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node199":{
"core":"testNodeLost_shard20_replica_n199",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10027_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node198":{
"core":"testNodeLost_shard20_replica_n198",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10073_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node242":{
"node_name":"127.0.0.1:10068_solr",
"core":"testNodeLost_shard20_replica_n241",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node197":{
"core":"testNodeLost_shard20_replica_n197",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10006_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node196":{
"core":"testNodeLost_shard20_replica_n196",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10098_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node195":{
"core":"testNodeLost_shard20_replica_n195",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10088_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node194":{
"core":"testNodeLost_shard20_replica_n194",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10079_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node192":{
"core":"testNodeLost_shard20_replica_n192",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10083_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"73330000-7fffffff",
"state":"active"},
"shard10":{
"replicas":{
"core_node92":{
"core":"testNodeLost_shard10_replica_n92",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10083_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node91":{
"core":"testNodeLost_shard10_replica_n91",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10064_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node94":{
"core":"testNodeLost_shard10_replica_n94",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10079_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node96":{
"core":"testNodeLost_shard10_replica_n96",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10098_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node95":{
"core":"testNodeLost_shard10_replica_n95",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10088_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node244":{
"node_name":"127.0.0.1:10046_solr",
"core":"testNodeLost_shard10_replica_n243",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node100":{
"core":"testNodeLost_shard10_replica_n100",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node98":{
"core":"testNodeLost_shard10_replica_n98",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10073_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node97":{
"core":"testNodeLost_shard10_replica_n97",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10006_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node99":{
"core":"testNodeLost_shard10_replica_n99",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10027_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"f3330000-ffffffff",
"state":"active"},
"shard11":{
"replicas":{
"core_node108":{
"core":"testNodeLost_shard11_replica_n108",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10075_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node107":{
"core":"testNodeLost_shard11_replica_n107",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10026_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node105":{
"core":"testNodeLost_shard11_replica_n105",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10042_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node104":{
"core":"testNodeLost_shard11_replica_n104",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10015_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node268":{
"node_name":"127.0.0.1:10044_solr",
"core":"testNodeLost_shard11_replica_n267",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node224":{
"node_name":"127.0.0.1:10078_solr",
"core":"testNodeLost_shard11_replica_n223",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node256":{
"node_name":"127.0.0.1:10000_solr",
"core":"testNodeLost_shard11_replica_n255",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node102":{
"core":"testNodeLost_shard11_replica_n102",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10031_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node110":{
"core":"testNodeLost_shard11_replica_n110",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node240":{
"node_name":"127.0.0.1:10063_solr",
"core":"testNodeLost_shard11_replica_n239",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"}},
"range":"0-ccbffff",
"state":"active"},
"shard12":{
"replicas":{
"core_node119":{
"core":"testNodeLost_shard12_replica_n119",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10067_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node118":{
"core":"testNodeLost_shard12_replica_n118",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10045_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node117":{
"core":"testNodeLost_shard12_replica_n117",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10028_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node116":{
"core":"testNodeLost_shard12_replica_n116",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10046_solr",
"state":"active",
"type":"NRT",
[...truncated too long message...]
file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1251643425
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1251643425
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1251643425
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null344577569
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null344577569
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null344577569
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 74ms :: artifacts dl 9ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 173 minutes 5 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 68 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/68/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([7D9D96624973904E]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([7D9D96624973904E]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([7D9D96624973904E:56624339D30B859E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13309 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> 2107038 INFO (SUITE-TestTriggerIntegration-seed#[7D9D96624973904E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_7D9D96624973904E-001/init-core-data-001
[junit4] 2> 2107040 INFO (SUITE-TestTriggerIntegration-seed#[7D9D96624973904E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2107041 INFO (SUITE-TestTriggerIntegration-seed#[7D9D96624973904E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2107117 DEBUG (SUITE-TestTriggerIntegration-seed#[7D9D96624973904E]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 2107118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2107119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 2107119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 2107119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 2107146 WARN (SUITE-TestTriggerIntegration-seed#[7D9D96624973904E]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2107148 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 2107149 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107149 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 2107157 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 2107158 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2107159 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2107159 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2107160 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2107160 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2107160 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2107160 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2107165 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2107165 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2107172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2107172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 2107173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 2107173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 2107196 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 2107206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 2107207 DEBUG (ScheduledTrigger-5080-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107227 DEBUG (ScheduledTrigger-5080-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107249 DEBUG (ScheduledTrigger-5080-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107270 DEBUG (ScheduledTrigger-5080-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107349 DEBUG (ScheduledTrigger-5080-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107379 DEBUG (ScheduledTrigger-5080-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107382 DEBUG (simCloudManagerPool-5079-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2107383 DEBUG (simCloudManagerPool-5079-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 2107383 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 2107384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 2107386 DEBUG (simCloudManagerPool-5079-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2107386 DEBUG (simCloudManagerPool-5079-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 2107386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 2107386 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2107386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 2107386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 2107387 DEBUG (ScheduledTrigger-5080-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 2107387 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10001_solr
[junit4] 2> 2107399 DEBUG (ScheduledTrigger-5080-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2107399 DEBUG (ScheduledTrigger-5080-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
[junit4] 2> 2107407 DEBUG (ScheduledTrigger-5080-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 2107407 DEBUG (ScheduledTrigger-5080-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
[junit4] 2> 2107419 DEBUG (ScheduledTrigger-5080-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2107428 DEBUG (ScheduledTrigger-5080-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 2107428 DEBUG (ScheduledTrigger-5080-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10000_solr]
[junit4] 2> 2107428 DEBUG (ScheduledTrigger-5080-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349c06a57ea42T1phqql6vugm98cpp64xniscey",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":27947113461049922,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947113461049922],
[junit4] 2> "nodeNames":["127.0.0.1:10000_solr"]}}
[junit4] 2> 2107428 DEBUG (ScheduledTrigger-5080-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 2107539 DEBUG (AutoscalingActionExecutor-5081-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349c06a57ea42T1phqql6vugm98cpp64xniscey",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":27947113461049922,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947113461049922],
[junit4] 2> "_enqueue_time_":27947114536680622,
[junit4] 2> "nodeNames":["127.0.0.1:10000_solr"]}}
[junit4] 2> 2107554 DEBUG (simCloudManagerPool-5079-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2107555 DEBUG (simCloudManagerPool-5079-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 2107555 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 2107555 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107555 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 2107556 DEBUG (AutoscalingActionExecutor-5081-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 2107556 DEBUG (AutoscalingActionExecutor-5081-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 2107556 DEBUG (AutoscalingActionExecutor-5081-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 874 ms for event id=6349c06a57ea42T1phqql6vugm98cpp64xniscey
[junit4] 2> 2107597 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
[junit4] 2> 2107610 INFO (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 1
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10000_solr
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 1
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 2107620 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 2107620 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 2107627 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 2107627 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2107627 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107627 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 2107628 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10000_solr
[junit4] 2> 2107628 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2107628 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 2107628 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2107629 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2107629 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2107629 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2107629 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2107629 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2107640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2107640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 2107641 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 2107641 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 2107648 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 2107649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 2107649 DEBUG (ScheduledTrigger-5083-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107698 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107719 DEBUG (ScheduledTrigger-5083-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107739 DEBUG (ScheduledTrigger-5083-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107747 DEBUG (simCloudManagerPool-5082-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2107747 DEBUG (simCloudManagerPool-5082-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 2107748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 2107748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2107748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 2107759 DEBUG (ScheduledTrigger-5083-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107827 DEBUG (ScheduledTrigger-5083-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107828 INFO (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 2107828 DEBUG (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2107828 DEBUG (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2107829 DEBUG (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 2107830 INFO (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 2107830 DEBUG (simCloudManagerPool-5082-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2107830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 2107830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 2107830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 2107830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 2107840 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2107840 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2107840 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 27947135117718622
[junit4] 2> 2107852 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2107860 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2107860 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2107860 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [27947135117718622], now=27947136135711572
[junit4] 2> 2107861 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 2107861 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 2107903 DEBUG (ScheduledTrigger-5083-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 2107903 DEBUG (AutoscalingActionExecutor-5084-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "_enqueue_time_":27947136175688872,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 2107904 INFO (AutoscalingActionExecutor-5084-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "_enqueue_time_":27947136175688872,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 2107904 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2107904 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2107904 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2107905 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2107905 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2107905 WARN (AutoscalingActionExecutor-5084-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 2107906 DEBUG (AutoscalingActionExecutor-5084-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 128 ms for event id=6349c5752e745eT1phqql6vugm98cpp64xniscf1
[junit4] 2> 2107906 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2107906 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2107906 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2107927 WARN (simCloudManagerPool-5082-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- can't find any active replicas for .system / shard1
[junit4] 2> 2107927 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 2107928 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 2107928 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2107928 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
[junit4] 2> 2107928 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 2107928 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2107928 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10002_solr
[junit4] 2> 2107940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2107940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 2107940 DEBUG (ScheduledTrigger-5086-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2107940 DEBUG (ScheduledTrigger-5086-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10002_solr
[junit4] 2> 2107940 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 2107941 DEBUG (ScheduledTrigger-5086-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "_dequeue_time_":27947140158478172,
[junit4] 2> "_enqueue_time_":27947136175688872,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 2107944 DEBUG (ScheduledTrigger-5086-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 2107972 DEBUG (ScheduledTrigger-5086-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2107972 DEBUG (ScheduledTrigger-5086-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2107972 DEBUG (AutoscalingActionExecutor-5087-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "_dequeue_time_":27947140158478172,
[junit4] 2> "_enqueue_time_":27947136175688872,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 2107972 INFO (AutoscalingActionExecutor-5087-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"6349c5752e745eT1phqql6vugm98cpp64xniscf1",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27947135117718622,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947135117718622],
[junit4] 2> "_dequeue_time_":27947140158478172,
[junit4] 2> "_enqueue_time_":27947136175688872,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 2107985 DEBUG (AutoscalingActionExecutor-5087-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 2107991 DEBUG (AutoscalingActionExecutor-5087-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 2107991 DEBUG (AutoscalingActionExecutor-5087-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 928 ms for event id=6349c5752e745eT1phqql6vugm98cpp64xniscf1
[junit4] 2> 2108028 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 2108028 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10002_solr
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 2
[junit4] 2> ## - update 3
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger1
[junit4] 2> ## - STARTED 2
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 2108032 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 2108032 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 2108032 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 2108033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2108033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 2108033 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10002_solr
[junit4] 2> 2108033 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2108033 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2108033 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2108034 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2108034 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2108034 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2108034 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2108034 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2108038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2108038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 2108038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
[junit4] 2> 2108047 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 2108053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2108079 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108079 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 2108087 DEBUG (ScheduledTrigger-5089-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108119 DEBUG (ScheduledTrigger-5089-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108139 DEBUG (ScheduledTrigger-5089-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108150 DEBUG (simCloudManagerPool-5088-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2108150 DEBUG (simCloudManagerPool-5088-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 2108150 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 2108151 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108151 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 2108161 DEBUG (simCloudManagerPool-5088-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2108161 DEBUG (simCloudManagerPool-5088-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 2108161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 2108161 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2108161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2108162 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108162 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 2108169 DEBUG (ScheduledTrigger-5089-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108169 DEBUG (ScheduledTrigger-5089-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 2108169 DEBUG (ScheduledTrigger-5089-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
[junit4] 2> 2108189 DEBUG (ScheduledTrigger-5089-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108189 DEBUG (ScheduledTrigger-5089-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 2108203 DEBUG (simCloudManagerPool-5088-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2108203 DEBUG (simCloudManagerPool-5088-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 2108204 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 2108204 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2108204 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2108204 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10004_solr
[junit4] 2> 2108205 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108205 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 2108205 DEBUG (ScheduledTrigger-5089-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 2108205 DEBUG (ScheduledTrigger-5089-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10004_solr]
[junit4] 2> 2108205 DEBUG (ScheduledTrigger-5089-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349c9b36e73dcT1phqql6vugm98cpp64xniscf5",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":27947153341969372,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947153341969372],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 2108206 DEBUG (ScheduledTrigger-5089-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_restore_trigger]
[junit4] 2> 2108418 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 2108420 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 1
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10004_solr
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_restore_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 2108496 DEBUG (AutoscalingActionExecutor-5090-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349c9b36e73dcT1phqql6vugm98cpp64xniscf5",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":27947153341969372,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947153341969372],
[junit4] 2> "_enqueue_time_":27947153396539272,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 2108516 DEBUG (AutoscalingActionExecutor-5090-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 2108516 DEBUG (AutoscalingActionExecutor-5090-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 100ms
[junit4] 2> 2108516 DEBUG (AutoscalingActionExecutor-5090-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1023 ms for event id=6349c9b36e73dcT1phqql6vugm98cpp64xniscf5
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeLostTriggerRestoreState -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-NL -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.49s J1 | TestTriggerIntegration.testNodeLostTriggerRestoreState <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
[junit4] > at __randomizedtesting.SeedInfo.seed([7D9D96624973904E:56624339D30B859E]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:322)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2108528 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventFromRestoredState
[junit4] 2> 2108528 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 2108528 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 2108529 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2108529 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108529 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 2108534 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10004_solr
[junit4] 2> 2108534 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2108534 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2108535 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2108536 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2108536 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 2108537 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion -1
[junit4] 2> 2108537 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 2108537 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2108537 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108537 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 2108548 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108575 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108595 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108615 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108635 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108656 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2108665 DEBUG (simCloudManagerPool-5091-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2108666 DEBUG (simCloudManagerPool-5091-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 2108666 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 2108667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2108667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 2108667 DEBUG (simCloudManagerPool-5091-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2108668 DEBUG (simCloudManagerPool-5091-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 2108668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 2108668 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2108668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2108668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 2108668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 2108668 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108669 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2108676 DEBUG (ScheduledTrigger-5092-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108689 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108689 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108689 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10005_solr at time 27947177559091572
[junit4] 2> 2108696 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108709 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108709 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108716 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108729 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108729 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108736 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108750 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108750 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108757 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108770 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108770 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108777 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108790 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108790 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108797 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108810 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108810 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108817 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108830 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108830 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108837 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108851 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108851 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108857 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108871 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108871 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108877 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2108891 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2108891 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2108891 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10005_solr] added at times [27947177559091572], now=27947187680000672
[junit4] 2> 2108891 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349cf56e26974T1phqql6vugm98cpp64xniscf8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27947177559091572,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947177559091572],
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 2108897 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 2108953 DEBUG (ScheduledTrigger-5092-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10005_solr
[junit4] 2> 2108953 DEBUG (AutoscalingActionExecutor-5093-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349cf56e26974T1phqql6vugm98cpp64xniscf8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27947177559091572,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947177559091572],
[junit4] 2> "_enqueue_time_":27947187962289922,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 2108954 DEBUG (AutoscalingActionExecutor-5093-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 2108954 DEBUG (AutoscalingActionExecutor-5093-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 2108954 DEBUG (AutoscalingActionExecutor-5093-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 69 ms for event id=6349cf56e26974T1phqql6vugm98cpp64xniscf8
[junit4] 2> 2109055 DEBUG (ScheduledTrigger-5092-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109055 DEBUG (ScheduledTrigger-5092-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109055 DEBUG (ScheduledTrigger-5092-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109055 DEBUG (ScheduledTrigger-5092-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10006_solr at time 27947195888590422
[junit4] 2> 2109084 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109084 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109084 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109104 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109104 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109104 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109124 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109124 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109124 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109144 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109144 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109144 DEBUG (ScheduledTrigger-5092-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109154 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2109154 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2109154 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2109154 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2109154 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2109155 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2109155 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2109155 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2109157 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 2109168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion -1
[junit4] 2> 2109168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 2109177 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2109182 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2109182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2109182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 2109182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10006_solr
[junit4] 2> 2109182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10006_solr
[junit4] 2> 2109182 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109188 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 2109188 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109188 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109216 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109216 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109216 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109236 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109236 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109237 DEBUG (ScheduledTrigger-5095-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2109257 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 2109257 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2109257 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10006_solr] added at times [27947195888590422], now=27947205957536872
[junit4] 2> 2109257 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349d39b685e56T1phqql6vugm98cpp64xniscfb",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27947195888590422,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947195888590422],
[junit4] 2> "nodeNames":["127.0.0.1:10006_solr"]}}
[junit4] 2> 2109257 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 2109273 DEBUG (ScheduledTrigger-5095-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10006_solr
[junit4] 2> 2109274 DEBUG (AutoscalingActionExecutor-5096-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6349d39b685e56T1phqql6vugm98cpp64xniscfb",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27947195888590422,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947195888590422],
[junit4] 2> "_enqueue_time_":27947205982660772,
[junit4] 2> "nodeNames":["127.0.0.1:10006_solr"]}}
[junit4] 2> 2109276 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventFromRestoredState
[junit4] 2> 2109277 INFO (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 4
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system]
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 2
[junit4] 2> ## - update 3
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 2
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 2109291 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 2109291 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 2109292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 2109292 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 2109292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2109292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2109292 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 2109423 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 2109423 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 2109423 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 2109424 WARN (AutoscalingActionExecutor-5096-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 2109425 DEBUG (AutoscalingActionExecutor-5096-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 7534 ms for event id=6349d39b685e56T1phqql6vugm98cpp64xniscfb
[junit4] 2> 2109425 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 2109425 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 2109425 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[7D9D96624973904E]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2109445 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2109447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 2109448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion -1
[junit4] 2> 2109448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 2109450 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2109450 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2109451 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 2109451 DEBUG (ScheduledTrigger-5098-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109471 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109492 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109512 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109532 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109553 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109573 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109574 DEBUG (simCloudManagerPool-5097-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2109574 DEBUG (simCloudManagerPool-5097-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 21
[junit4] 2> 2109577 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 21
[junit4] 2> 2109577 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2109577 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
[junit4] 2> 2109593 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2109594 DEBUG (simCloudManagerPool-5097-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2109594 DEBUG (simCloudManagerPool-5097-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 22
[junit4] 2> 2109595 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 22
[junit4] 2> 2109595 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2109595 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
[junit4] 2> 2109595 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 2109595 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 22, lastZnodeVersion 22
[junit4] 2> 2109595 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2109595 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2109613 DEBUG (ScheduledTrigger-5098-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2109615 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2109615 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2109615 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10007_solr at time 27947223886925172
[junit4] 2> 2109621 WARN (MetricsHistoryHandler-5078-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
[junit4] 2> 2109634 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2109636 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2109636 DEBUG (ScheduledTrigger-5098-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2109640 DEBUG (simCloudManagerPool-5097-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2109641 DEBUG (simCloudManagerPool-5097-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 23
[junit4] 2> 2109641 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 23
[junit4] 2> 2109642 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 2109642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr, 127.0.0.1:10007_solr]
[junit4] 2> 2109642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 2109642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10007_solr
[junit4] 2> 2109642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10007_solr
[junit4] 2> 2109642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 23, lastZnodeVersion 23
[junit4] 2> 2109642 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2109642 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2109642 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10007_solr] added at times [27947225212602872], now=27947225226398972
[junit4] 2> 2109642 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6349da6f4149f8T1phqql6vugm98cpp64xniscfd",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":27947225212602872,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27947225212602872],
[junit4] 2> "nodeNames":["127.0.0.1:10007_solr"]}}
[junit4] 2> 2109643 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 2109691 DEBUG (ScheduledTrigger-5098-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10007_solr
[junit4] 2> 2109691 DEBUG (AutoscalingActionExecutor-5099-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing
[...truncated too long message...]
e = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1187052481
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1187052481
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1187052481
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1003739714
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1003739714
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1003739714
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 79ms :: artifacts dl 6ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 287 minutes 18 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 67 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/67/
6 tests failed.
FAILED: org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaOnIndexing
Error Message:
Could not load collection from ZK: deleteReplicaOnIndexing
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: deleteReplicaOnIndexing
at __randomizedtesting.SeedInfo.seed([D8C8FE9657F440E4:A1B3D219F4698AC7]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1302)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:731)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:117)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:256)
at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaOnIndexing(DeleteReplicaTest.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/deleteReplicaOnIndexing/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1314)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1300)
... 45 more
FAILED: org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
Error Message:
Error from server at https://127.0.0.1:41559/solr: KeeperErrorCode = Session expired for /configs/conf
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:41559/solr: KeeperErrorCode = Session expired for /configs/conf
at __randomizedtesting.SeedInfo.seed([D8C8FE9657F440E4:B2DE9F463F060A2E]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:235)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
103 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 1) Thread[id=2270, name=Thread-461, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2967, name=Thread-440-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 3) Thread[id=2190, name=Thread-404, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=2281, name=Thread-472, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=2201, name=Thread-415, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=2186, name=Thread-400, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2284, name=Thread-475, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=2303, name=Thread-494, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=2279, name=Thread-470, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=2296, name=Thread-487, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=2301, name=Thread-492, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=2302, name=Thread-493, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=2252, name=Thread-443, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=2244, name=Thread-435, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=2208, name=Thread-422, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=2304, name=Thread-495, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=2175, name=Thread-394, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=2254, name=Thread-445, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=2193, name=Thread-407, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=2205, name=Thread-419, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=2263, name=Thread-454, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=2297, name=Thread-488, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=2195, name=Thread-409, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=2271, name=Thread-462, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=2291, name=Thread-482, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=2257, name=Thread-448, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log4jLogger.java:309) at org.apache.solr.cloud.DeleteReplicaTest.lambda$deleteReplicaOnIndexing$11(DeleteReplicaTest.java:375) at org.apache.solr.cloud.DeleteReplicaTest$$Lambda$555/795985998.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=2275, name=Thread-466, state=BLOCKED, group=TGRP-DeleteReplicaTest] at org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:361) at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:96) at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:65) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68) at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220) at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170) at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2025) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1898) at org.apache.logging.slf4j.Log4jLogger.error(Log
[...truncated too long message...]
-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null626619960
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null626619960
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null626619960
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1725888443
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1725888443
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1725888443
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 53ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 262 minutes 21 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 66 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/66/
4 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling
Error Message:
Both triggers should have fired by now
Stack Trace:
java.lang.AssertionError: Both triggers should have fired by now
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5:2224F5B0BD6EC27]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5:A448F9F716BAA9FA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12499 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> 268300 INFO (SUITE-TestTriggerIntegration-seed#[F900E77ED97C0FB5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F900E77ED97C0FB5-001/init-core-data-001
[junit4] 2> 268303 INFO (SUITE-TestTriggerIntegration-seed#[F900E77ED97C0FB5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 268304 INFO (SUITE-TestTriggerIntegration-seed#[F900E77ED97C0FB5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 268334 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 268337 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 268337 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 268343 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 268373 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 268374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 268374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 268401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 268421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 268421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 268429 DEBUG (ScheduledTrigger-460-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268477 DEBUG (ScheduledTrigger-460-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268497 DEBUG (ScheduledTrigger-460-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268589 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268610 DEBUG (ScheduledTrigger-460-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268616 DEBUG (simCloudManagerPool-459-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 268616 DEBUG (simCloudManagerPool-459-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 268617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 268618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 268618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 268630 DEBUG (ScheduledTrigger-460-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268642 DEBUG (simCloudManagerPool-459-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 268643 DEBUG (simCloudManagerPool-459-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 268643 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 268644 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 268644 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 268644 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 268645 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 268645 DEBUG (ScheduledTrigger-460-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 268645 DEBUG (ScheduledTrigger-460-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 268650 DEBUG (ScheduledTrigger-460-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 268666 DEBUG (ScheduledTrigger-460-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 268666 DEBUG (ScheduledTrigger-460-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 268666 DEBUG (ScheduledTrigger-460-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 27755969227385018
[junit4] 2> 268671 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 268687 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 268687 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 268687 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [27755969227385018], now=27755970263101018
[junit4] 2> 268690 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629be82e31e8baT9o1908pm3zty2i97uyhhmegso",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27755969227385018,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755969227385018],
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 268691 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 268717 DEBUG (ScheduledTrigger-460-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 268717 DEBUG (AutoscalingActionExecutor-461-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629be82e31e8baT9o1908pm3zty2i97uyhhmegso",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":27755969227385018,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755969227385018],
[junit4] 2> "_enqueue_time_":27755970478059468,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 268759 DEBUG (AutoscalingActionExecutor-461-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 268759 DEBUG (AutoscalingActionExecutor-461-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 268759 DEBUG (AutoscalingActionExecutor-461-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2089 ms for event id=629be82e31e8baT9o1908pm3zty2i97uyhhmegso
[junit4] 2> 268766 DEBUG (simCloudManagerPool-459-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 268768 DEBUG (simCloudManagerPool-459-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 268774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 268774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 268815 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 268820 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 268827 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 268827 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 268827 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 268827 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 268828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 268828 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 268832 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 268833 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 268835 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 268835 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 268836 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 268836 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 268836 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 268836 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 268848 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 268848 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 268852 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 268852 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 268867 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 268876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 268876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 268877 DEBUG (ScheduledTrigger-463-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268907 DEBUG (ScheduledTrigger-463-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268927 DEBUG (ScheduledTrigger-463-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268961 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 268973 DEBUG (simCloudManagerPool-462-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 268973 DEBUG (simCloudManagerPool-462-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 268974 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 268974 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 268974 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 269007 DEBUG (ScheduledTrigger-463-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269016 INFO (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 269016 DEBUG (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 269017 DEBUG (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 269017 DEBUG (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 269018 INFO (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 269018 DEBUG (simCloudManagerPool-462-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 269018 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 269019 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269019 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 269019 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10003_solr
[junit4] 2> 269019 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 269019 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 269019 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269019 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 269027 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 269039 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269039 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 269040 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [27755986852677368], now=27755987885954818
[junit4] 2> 269040 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 269040 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 269149 DEBUG (ScheduledTrigger-463-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 269149 DEBUG (AutoscalingActionExecutor-464-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "_enqueue_time_":27755987917640618,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 269149 INFO (AutoscalingActionExecutor-464-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "_enqueue_time_":27755987917640618,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 269153 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 269154 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 269154 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 269154 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 269154 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 269154 WARN (AutoscalingActionExecutor-464-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:202) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 269160 DEBUG (AutoscalingActionExecutor-464-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 546 ms for event id=629bec48be86f8T9o1908pm3zty2i97uyhhmegsr
[junit4] 2> 269160 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 269160 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 269160 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 269161 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 269168 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 269168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 269168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion -1
[junit4] 2> 269168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 269172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr
[junit4] 2> 269191 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269191 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 269198 DEBUG (ScheduledTrigger-466-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269198 DEBUG (ScheduledTrigger-466-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 269209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 269209 DEBUG (ScheduledTrigger-466-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "_dequeue_time_":27755996361697518,
[junit4] 2> "_enqueue_time_":27755987917640618,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 269216 DEBUG (ScheduledTrigger-466-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 269243 DEBUG (ScheduledTrigger-466-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269243 DEBUG (ScheduledTrigger-466-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 269244 DEBUG (AutoscalingActionExecutor-467-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "_dequeue_time_":27755996361697518,
[junit4] 2> "_enqueue_time_":27755987917640618,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 269244 INFO (AutoscalingActionExecutor-467-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"629bec48be86f8T9o1908pm3zty2i97uyhhmegsr",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27755986852677368,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27755986852677368],
[junit4] 2> "_dequeue_time_":27755996361697518,
[junit4] 2> "_enqueue_time_":27755987917640618,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 269259 DEBUG (AutoscalingActionExecutor-467-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 269259 DEBUG (AutoscalingActionExecutor-467-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 269259 DEBUG (AutoscalingActionExecutor-467-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 776 ms for event id=629bec48be86f8T9o1908pm3zty2i97uyhhmegsr
[junit4] 2> 269261 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 269262 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10001_solr
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 2
[junit4] 2> ## - update 3
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger1
[junit4] 2> ## - STARTED 2
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 269264 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 269264 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 269264 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10001_solr
[junit4] 2> 269264 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 269264 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 269265 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 269265 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 269265 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 269265 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 269273 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 269273 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 269279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 269282 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 269283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion -1
[junit4] 2> 269283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 269292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269305 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 269305 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 269306 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269326 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269347 DEBUG (ScheduledTrigger-469-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269367 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269386 DEBUG (simCloudManagerPool-468-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 269386 DEBUG (simCloudManagerPool-468-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 269387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 269387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 269387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 269387 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269389 DEBUG (simCloudManagerPool-468-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 269389 DEBUG (simCloudManagerPool-468-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 269390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 269390 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 269390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 269390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 269392 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269392 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 269401 DEBUG (simCloudManagerPool-468-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 269401 DEBUG (simCloudManagerPool-468-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 269407 DEBUG (ScheduledTrigger-469-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 269413 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 269413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 269413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 269413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 269413 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 269413 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 269414 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269414 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 269428 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 27756007610007118
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [27756007610007118], now=27756007612541218
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bf11dfa3e4eT9o1908pm3zty2i97uyhhmegsv",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27756007610007118,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27756007610007118],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 269434 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 269435 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 269435 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 269435 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 27756007659844768
[junit4] 2> 269435 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [27756007659844768], now=27756007662010118
[junit4] 2> 269435 DEBUG (ScheduledTrigger-469-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bf120f2b4a0T9o1908pm3zty2i97uyhhmegsx",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":27756007659844768,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27756007659844768],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 269462 DEBUG (ScheduledTrigger-469-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10004_solr
[junit4] 2> 269463 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629bf11dfa3e4eT9o1908pm3zty2i97uyhhmegsv",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":27756007610007118,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27756007610007118],
[junit4] 2> "_enqueue_time_":27756007638461818,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 269463 INFO (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 269483 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 269484 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 269484 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 269485 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1092 ms for event id=629bf11dfa3e4eT9o1908pm3zty2i97uyhhmegsv
[junit4] 2> 269841 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 269842 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 269842 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 269842 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [27756007659844768], now=27756028006776018
[junit4] 2> 269842 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bf120f2b4a0T9o1908pm3zty2i97uyhhmegt0",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":27756007659844768,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27756007659844768],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 269842 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 269848 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 269848 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 3
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger1
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2> ## * Trigger: node_added_trigger2
[junit4] 2> ## - IGNORED 1
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testTriggerThrottling -Dtests.seed=F900E77ED97C0FB5 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=lv -Dtests.timezone=Brazil/East -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.59s J0 | TestTriggerIntegration.testTriggerThrottling <<<
[junit4] > Throwable #1: java.lang.AssertionError: Both triggers should have fired by now
[junit4] > at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5:2224F5B0BD6EC27]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 269861 DEBUG (ScheduledTrigger-469-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10004_solr
[junit4] 2> 269861 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629bf120f2b4a0T9o1908pm3zty2i97uyhhmegt0",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":27756007659844768,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[27756007659844768],
[junit4] 2> "_enqueue_time_":27756028027849518,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 269862 INFO (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 27756009064899318 time = 27756028989238218
[junit4] 2> 269862 INFO (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 269876 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 269876 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 269876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 269876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 269876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 269876 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 269880 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 269880 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 269880 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 269880 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 269880 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 269881 WARN (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:540) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:756) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:321) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:202) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:538) ~[test/:?]
[junit4] 2> ... 10 more
[junit4] 2> 269881 DEBUG (AutoscalingActionExecutor-470-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1003 ms for event id=629bf120f2b4a0T9o1908pm3zty2i97uyhhmegt0
[junit4] 2> 269882 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 269882 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 269882 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 269887 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 269887 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 269888 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion -1
[junit4] 2> 269888 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 269900 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 269905 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 269905 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 269905 DEBUG (ScheduledTrigger-472-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269926 DEBUG (ScheduledTrigger-472-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269946 DEBUG (ScheduledTrigger-472-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269966 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 269986 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270000 DEBUG (simCloudManagerPool-471-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 270001 DEBUG (simCloudManagerPool-471-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 270003 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 270003 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 270003 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 270007 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270027 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270047 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270067 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270088 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270108 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270128 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270152 DEBUG (simCloudManagerPool-471-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"}}
[junit4] 2> 270149 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270173 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270193 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270206 DEBUG (simCloudManagerPool-471-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 270206 DEBUG (simCloudManagerPool-471-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 270214 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 270214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 270214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 270234 DEBUG (ScheduledTrigger-472-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270248 DEBUG (simCloudManagerPool-471-thread-7) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 270249 DEBUG (simCloudManagerPool-471-thread-7) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 270253 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 270254 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 270254 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 270255 DEBUG (ScheduledTrigger-472-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270275 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270295 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270316 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270336 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270356 DEBUG (ScheduledTrigger-472-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270377 DEBUG (ScheduledTrigger-472-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270379 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"629bfad96c9c9aT9o1908pm3zty2i97uyhhmegt3",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":27756049409547418,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"collection1":{"shard1":500.0}},
[junit4] 2> "hotCollections":{"collection1":500.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 270381 DEBUG (ScheduledTrigger-472-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 270450 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"629bfad96c9c9aT9o1908pm3zty2i97uyhhmegt3",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":27756049409547418,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"collection1":{"shard1":500.0}},
[junit4] 2> "hotCollections":{"collection1":500.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "_enqueue_time_":27756054955468118,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 270450 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"629bfad96c9c9aT9o1908pm3zty2i97uyhhmegt3",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":27756049409547418,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"collection1":{"shard1":500.0}},
[junit4] 2> "hotCollections":{"collection1":500.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "_enqueue_time_":27756054955468118,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 270477 INFO (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
[junit4] 2> 270478 INFO (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 270490 INFO (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 270490 INFO (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 27756060431 sessionWrapper.createTime 27756058862051068, this.sessionWrapper.createTime 27756058862051068
[junit4] 2> 270492 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"629bfad96c9c9aT9o1908pm3zty2i97uyhhmegt3",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":27756049409547418,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "__start__":3,
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"collection1":{"shard1":500.0}},
[junit4] 2> "hotCollections":{"collection1":500.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "_enqueue_time_":27756054955468118,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"collection1\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@4e6d734d, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@2f67221b, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@55a6c02c], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 270497 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
[junit4] 2> 270506 INFO (simCloudManagerPool-471-thread-9) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
[junit4] 2> 270549 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 270578 INFO (simCloudManagerPool-471-thread-13) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10004_solr for creating new replica
[junit4] 2> 270589 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 270719 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 270749 INFO (simCloudManagerPool-471-thread-17) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10004_solr for creating new replica
[junit4] 2> 270777 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 270777 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: search_rate_trigger after 100ms
[junit4] 2> 270777 DEBUG (AutoscalingActionExecutor-473-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 16359 ms for event id=629bfad96c9c9aT9o1908pm3zty2i97uyhhmegt3
[junit4] 2> 270788 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [collection1]
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 4
[junit4] 2> ## * collection1 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - ADDREPLICA 3
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETESTATUS 3
[junit4] 2> ## - REQUESTSTATUS 3
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: search_rate_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=F900E77ED97C0FB5 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=lv -Dtests.timezone=Brazil/East -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.94s J0 | TestTriggerIntegration.testSearchRate <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
[junit4] > at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5:A448F9F716BAA9FA]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 270806 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 270806 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 270807 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 270807 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 270807 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 270807 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
[junit4] 2> 270807 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 270807 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 270808 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 270808 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 270809 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 270809 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 270809 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 270809 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F900E77ED97C0FB5]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 270819 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 270820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 270820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion -1
[junit4] 2> 270820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 270830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 270838 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 270838 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 270838 DEBUG (ScheduledTrigger-475-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270873 DEBUG (ScheduledTrigger-475-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270893 DEBUG (ScheduledTrigger-475-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270913 DEBUG (ScheduledTrigger-475-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270920 DEBUG (simCloudManagerPool-474-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 270934 DEBUG (ScheduledTrigger-475-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 270937 DEBUG (simCloudManagerPool-474-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version
[...truncated too long message...]
file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2013265382
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2013265382
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2013265382
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null216736933
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null216736933
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null216736933
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 59ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 284 minutes 11 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 65 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/65/
5 tests failed.
FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings
Error Message:
pos=-1 nextPos=1 count=1
Stack Trace:
java.lang.AssertionError: pos=-1 nextPos=1 count=1
at __randomizedtesting.SeedInfo.seed([66E54A55B5221BCF:CBEF544EC6C3B3C]:0)
at org.apache.lucene.util.RollingBuffer.get(RollingBuffer.java:105)
at org.apache.lucene.analysis.core.FlattenGraphFilter.incrementToken(FlattenGraphFilter.java:279)
at org.apache.lucene.analysis.miscellaneous.ConditionalTokenFilter.incrementToken(ConditionalTokenFilter.java:178)
at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:65)
at org.apache.lucene.analysis.payloads.DelimitedPayloadTokenFilter.incrementToken(DelimitedPayloadTokenFilter.java:55)
at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:65)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:441)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:882)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:60848/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:60666/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:60848/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:60666/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([F24B3A2F5BFBCD30:5886E9DDEC2818E0]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60848/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 46 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
waitFor not elapsed but produced an event
Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
at __randomizedtesting.SeedInfo.seed([F24B3A2F5BFBCD30:91800CADC234BE1D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/107)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1527427088527382350", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":25, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":25}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1527427088560603750", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":12, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":12}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":12, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":12}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1527427088560267200", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":13, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":13}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":13, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":13}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/107)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1527427088527382350",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":25,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":25}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1527427088560603750",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":12,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":12},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":12,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":12}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1527427088560267200",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":13,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":13},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":13,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":13}}}}}
at __randomizedtesting.SeedInfo.seed([F24B3A2F5BFBCD30:CBC5836F740404CE]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:312)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([F24B3A2F5BFBCD30:AF0324A6943D6B7F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 2651 lines...]
[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
[junit4] 2> Exception from random analyzer:
[junit4] 2> charfilters=
[junit4] 2> org.apache.lucene.analysis.fa.PersianCharFilter(java.io.StringReader@4fa58c49)
[junit4] 2> org.apache.lucene.analysis.charfilter.MappingCharFilter(org.apache.lucene.analysis.charfilter.NormalizeCharMap@406ecb2b, org.apache.lucene.analysis.fa.PersianCharFilter@671706c4)
[junit4] 2> tokenizer=
[junit4] 2> org.apache.lucene.analysis.core.LetterTokenizer()
[junit4] 2> filters=
[junit4] 2> org.apache.lucene.analysis.miscellaneous.TypeAsSynonymFilter(ValidatingTokenFilter@1efd4576 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1)
[junit4] 2> org.apache.lucene.analysis.sv.SwedishLightStemFilter(ValidatingTokenFilter@7702e0ec term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false)ConditionalTokenFilter:
[junit4] 2> org.apache.lucene.analysis.core.FlattenGraphFilter(OneTimeWrapper@1f427f19 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false)
[junit4] 2> org.apache.lucene.analysis.payloads.DelimitedPayloadTokenFilter(ValidatingTokenFilter@308465ea term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false,payload=null, , org.apache.lucene.analysis.payloads.IdentityEncoder@59027ded)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=66E54A55B5221BCF -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ru-RU -Dtests.timezone=Africa/Djibouti -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.02s J2 | TestRandomChains.testRandomChainsWithLargeStrings <<<
[junit4] > Throwable #1: java.lang.AssertionError: pos=-1 nextPos=1 count=1
[junit4] > at __randomizedtesting.SeedInfo.seed([66E54A55B5221BCF:CBEF544EC6C3B3C]:0)
[junit4] > at org.apache.lucene.util.RollingBuffer.get(RollingBuffer.java:105)
[junit4] > at org.apache.lucene.analysis.core.FlattenGraphFilter.incrementToken(FlattenGraphFilter.java:279)
[junit4] > at org.apache.lucene.analysis.miscellaneous.ConditionalTokenFilter.incrementToken(ConditionalTokenFilter.java:178)
[junit4] > at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:65)
[junit4] > at org.apache.lucene.analysis.payloads.DelimitedPayloadTokenFilter.incrementToken(DelimitedPayloadTokenFilter.java:55)
[junit4] > at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:65)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:441)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
[junit4] > at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:882)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@634b6522), locale=ru-RU, timezone=Africa/Djibouti
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=125454112,total=204996608
[junit4] 2> NOTE: All tests run in this JVM: [DelimitedTermFrequencyTokenFilterTest, TestPatternReplaceFilter, TestSoraniAnalyzer, TestSynonymFilterFactory, TestUAX29URLEmailTokenizer, TestKeywordRepeatFilter, TestNorwegianMinimalStemFilterFactory, TestUnicodeWhitespaceTokenizer, TestAllAnalyzersHaveFactories, TestDecimalDigitFilterFactory, TestHunspellStemFilter, TestConv, TestCapitalizationFilterFactory, TestTwoFold, TestSimplePatternSplitTokenizer, TestTrimFilter, TestFlagNum, TestScandinavianFoldingFilterFactory, TestDelimitedPayloadTokenFilterFactory, TestProtectedTermFilter, TestPatternReplaceCharFilter, TestRussianAnalyzer, TestPortugueseAnalyzer, TestKStemFilterFactory, TestKStemmer, TestShingleFilterFactory, TestStandardFactories, TestCharArrayIterator, TestHindiAnalyzer, TestCJKWidthFilterFactory, TestTruncateTokenFilter, TestFinnishLightStemFilterFactory, TestRandomChains]
[junit4] Completed [85/289 (1!)] on J2 in 7.39s, 2 tests, 1 failure <<< FAILURES!
[...truncated 10204 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_F24B3A2F5BFBCD30-001/init-core-data-001
[junit4] 2> 751704 WARN (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=39 numCloses=39
[junit4] 2> 751704 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 751717 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 751718 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 751718 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_F24B3A2F5BFBCD30-001/tempDir-001
[junit4] 2> 751718 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 751726 INFO (Thread-903) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 751726 INFO (Thread-903) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 751825 ERROR (Thread-903) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 751826 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:60892
[junit4] 2> 751877 INFO (zkConnectionManagerCallback-1302-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 751907 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 751911 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 752053 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 752053 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 752053 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 752054 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53927c97{/solr,null,AVAILABLE}
[junit4] 2> 752054 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79959ad6{SSL,[ssl, http/1.1]}{127.0.0.1:56682}
[junit4] 2> 752055 INFO (jetty-launcher-1299-thread-1) [ ] o.e.j.s.Server Started @752122ms
[junit4] 2> 752055 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56682}
[junit4] 2> 752055 ERROR (jetty-launcher-1299-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 752055 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 752055 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 752056 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 752056 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 752056 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-27T13:15:31.930Z
[junit4] 2> 752135 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 752135 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 752135 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 752145 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c2c2d43{/solr,null,AVAILABLE}
[junit4] 2> 752164 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@57464944{SSL,[ssl, http/1.1]}{127.0.0.1:47233}
[junit4] 2> 752164 INFO (jetty-launcher-1299-thread-2) [ ] o.e.j.s.Server Started @752232ms
[junit4] 2> 752164 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47233}
[junit4] 2> 752165 ERROR (jetty-launcher-1299-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 752165 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 752165 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 752165 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 752165 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 752165 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-27T13:15:32.039Z
[junit4] 2> 752172 INFO (zkConnectionManagerCallback-1304-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752173 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 752209 INFO (jetty-launcher-1299-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60892/solr
[junit4] 2> 752231 INFO (zkConnectionManagerCallback-1306-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752231 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 752240 INFO (zkConnectionManagerCallback-1310-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752253 INFO (jetty-launcher-1299-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60892/solr
[junit4] 2> 752255 INFO (zkConnectionManagerCallback-1316-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752273 INFO (zkConnectionManagerCallback-1314-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752320 INFO (zkConnectionManagerCallback-1318-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752331 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 752332 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47233_solr
[junit4] 2> 752332 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.Overseer Overseer (id=73871553752530950-127.0.0.1:47233_solr-n_0000000000) starting
[junit4] 2> 752421 INFO (zkConnectionManagerCallback-1327-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 752425 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60892/solr ready
[junit4] 2> 752435 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 752436 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56682_solr
[junit4] 2> 752454 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 752454 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47233_solr
[junit4] 2> 752609 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 752610 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 752611 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 752611 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 752877 INFO (zkCallback-1326-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 752921 INFO (zkCallback-1315-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 752929 INFO (OverseerStateUpdate-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 752931 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:56682_solr, 127.0.0.1:47233_solr]
[junit4] 2> 752946 DEBUG (ScheduledTrigger-1697-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 752953 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 752953 DEBUG (OverseerAutoScalingTriggerThread-73871553752530950-127.0.0.1:47233_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 753289 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56682.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753335 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56682.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753335 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56682.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753336 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_F24B3A2F5BFBCD30-001/tempDir-001/node1/.
[junit4] 2> 753416 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47233.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753421 INFO (zkConnectionManagerCallback-1332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 753422 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 753423 INFO (jetty-launcher-1299-thread-1) [n:127.0.0.1:56682_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60892/solr ready
[junit4] 2> 753567 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47233.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753567 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47233.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@32f6e7f0
[junit4] 2> 753569 INFO (jetty-launcher-1299-thread-2) [n:127.0.0.1:47233_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_F24B3A2F5BFBCD30-001/tempDir-001/node2/.
[junit4] 2> 753693 INFO (zkConnectionManagerCallback-1334-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 753736 INFO (zkConnectionManagerCallback-1339-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 753737 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 753738 INFO (SUITE-IndexSizeTriggerTest-seed#[F24B3A2F5BFBCD30]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60892/solr ready
[junit4] 2> 753789 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 753789 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 753790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 753790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 753813 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 753837 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 753837 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 753837 DEBUG (ScheduledTrigger-1706-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 753867 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 753888 DEBUG (ScheduledTrigger-1706-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 753908 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 753928 DEBUG (ScheduledTrigger-1706-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 753995 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754000 DEBUG (ScheduledTrigger-1697-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754016 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754023 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
[junit4] 2> 754037 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754057 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754077 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754097 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754117 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754140 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754161 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754177 DEBUG (simCloudManagerPool-1705-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testTrigger_collection"}}
[junit4] 2> 754181 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754201 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754221 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754241 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754262 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754282 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754302 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754322 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754343 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754363 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754384 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754404 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754424 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754444 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754545 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754565 DEBUG (ScheduledTrigger-1706-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 754576 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [testTrigger_collection]
[junit4] 2> DocCollection(testTrigger_collection//clusterstate.json/28)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 4
[junit4] 2> ## * testTrigger_collection 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - update 26
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 754577 DEBUG (TEST-IndexSizeTriggerTest.testTrigger-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 754585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 754585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 754585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 754585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 754686 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testTrigger -Dtests.seed=F24B3A2F5BFBCD30 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-MT -Dtests.timezone=Africa/Bangui -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.69s J1 | IndexSizeTriggerTest.testTrigger <<<
[junit4] > Throwable #1: java.lang.AssertionError: waitFor not elapsed but produced an event
[junit4] > at __randomizedtesting.SeedInfo.seed([F24B3A2F5BFBCD30:91800CADC234BE1D]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:181)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 754769 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMixedBounds
[junit4] 2> 754769 WARN (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.c.a.IndexSizeTriggerTest Requires SOLR-12208
[junit4] 2> 754769 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 754769 DEBUG (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 754769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 754770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 754770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 754770 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 754870 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMixedBounds
[junit4] 2> 754912 INFO (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[F24B3A2F5BFBCD30]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMergeIntegration
[junit4] 2> 755068 DEBUG (ScheduledTrigger-1697-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 755286 DEBUG (simCloudManagerPool-1705-thread-39) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testMergeIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 755300 DEBUG (simCloudManagerPool-1705-thread-40) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testMergeIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 755454 DEBUG (simCloudManagerPool-1705-thread-52) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 755455 DEBUG (simCloudManagerPool-1705-thread-52) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 755455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 755455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 755455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 755455 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 755482 DEBUG (simCloudManagerPool-1705-thread-53) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 755482 DEBUG (simCloudManagerPool-1705-thread-53) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 755482 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 755483 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 755483 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 755483 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 755506 DEBUG (simCloudManagerPool-1705-thread-54) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 755506 DEBUG (simCloudManagerPool-1705-thread-54) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 755506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 755506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 755506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 755506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 755610 DEBUG (simCloudManagerPool-1705-thread-61) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 755610 DEBUG (simCloudManagerPool-1705-thread-61) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 755610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 755611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 755611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 755611 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 755736 DEBUG (ScheduledTrigger-1706-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 755737 DEBUG (ScheduledTrigger-1706-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger3]
[junit4] 2> 755739 INFO (ScheduledTrigger-1706-thread-2) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27678930423336417, stage=STARTED, actionName='null', event={
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
[junit4] 2> "trigger":"index_size_trigger3",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 755751 DEBUG (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 755752 INFO (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27678931114276567, stage=BEFORE_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger3}, config={
[junit4] 2> "trigger":"index_size_trigger3",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 755753 DEBUG (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 755755 INFO (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed plan empty, remained 0 requested ops to try.
[junit4] 2> 755755 INFO (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 27678931281 sessionWrapper.createTime 27678931177716567, this.sessionWrapper.createTime 27678931177716567
[junit4] 2> 755756 INFO (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27678931284464817, stage=AFTER_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger3, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger3",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 755756 INFO (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27678931307757617, stage=BEFORE_ACTION, actionName='execute_plan', event={
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger3, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger3",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 755756 DEBUG (AutoscalingActionExecutor-1707-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"6255d5b429624dT8a2ejqm18nb10bjcugmdemaio",
[junit4] 2> "source":"index_size_trigger3",
[junit4] 2> "eventTime":27678924056650317,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":27678930365356667,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {AFTER_ACTION=[compute_plan], unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4]
[...truncated too long message...]
:: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null265238557
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null265238557
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null265238557
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null658801087
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null658801087
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null658801087
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 78ms :: artifacts dl 6ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 176 minutes 44 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 64 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/64/
9 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Didn't see all replicas for shard shard1 in c8n_1x3 come up within 90000 ms! ClusterState: { "collMinRf_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"collMinRf_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:35996/_bvx/io", "node_name":"127.0.0.1:35996__bvx%2Fio", "state":"active", "type":"NRT", "leader":"true"}, "core_node5":{ "core":"collMinRf_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:38507/_bvx/io", "node_name":"127.0.0.1:38507__bvx%2Fio", "state":"active", "type":"NRT"}, "core_node6":{ "core":"collMinRf_1x3_shard1_replica_n3", "base_url":"http://127.0.0.1:39428/_bvx/io", "node_name":"127.0.0.1:39428__bvx%2Fio", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{"core_node44":{ "core":"collection1_shard1_replica_n43", "base_url":"http://127.0.0.1:35996/_bvx/io", "node_name":"127.0.0.1:35996__bvx%2Fio", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node42":{ "core":"collection1_shard2_replica_n41", "base_url":"http://127.0.0.1:39428/_bvx/io", "node_name":"127.0.0.1:39428__bvx%2Fio", "state":"active", "type":"NRT", "leader":"true"}, "core_node46":{ "core":"collection1_shard2_replica_n45", "base_url":"http://127.0.0.1:41305/_bvx/io", "node_name":"127.0.0.1:41305__bvx%2Fio", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node2":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:38507/_bvx/io", "node_name":"127.0.0.1:38507__bvx%2Fio", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "c8n_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"c8n_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:39428/_bvx/io", "node_name":"127.0.0.1:39428__bvx%2Fio", "state":"active", "type":"NRT", "leader":"true"}, "core_node5":{ "core":"c8n_1x3_shard1_replica_n3", "base_url":"http://127.0.0.1:41305/_bvx/io", "node_name":"127.0.0.1:41305__bvx%2Fio", "state":"recovering", "type":"NRT"}, "core_node6":{ "core":"c8n_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:35996/_bvx/io", "node_name":"127.0.0.1:35996__bvx%2Fio", "state":"down", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in c8n_1x3 come up within 90000 ms! ClusterState: {
"collMinRf_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"collMinRf_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:35996/_bvx/io",
"node_name":"127.0.0.1:35996__bvx%2Fio",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node5":{
"core":"collMinRf_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:38507/_bvx/io",
"node_name":"127.0.0.1:38507__bvx%2Fio",
"state":"active",
"type":"NRT"},
"core_node6":{
"core":"collMinRf_1x3_shard1_replica_n3",
"base_url":"http://127.0.0.1:39428/_bvx/io",
"node_name":"127.0.0.1:39428__bvx%2Fio",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node44":{
"core":"collection1_shard1_replica_n43",
"base_url":"http://127.0.0.1:35996/_bvx/io",
"node_name":"127.0.0.1:35996__bvx%2Fio",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node42":{
"core":"collection1_shard2_replica_n41",
"base_url":"http://127.0.0.1:39428/_bvx/io",
"node_name":"127.0.0.1:39428__bvx%2Fio",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node46":{
"core":"collection1_shard2_replica_n45",
"base_url":"http://127.0.0.1:41305/_bvx/io",
"node_name":"127.0.0.1:41305__bvx%2Fio",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:38507/_bvx/io",
"node_name":"127.0.0.1:38507__bvx%2Fio",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"c8n_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"c8n_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:39428/_bvx/io",
"node_name":"127.0.0.1:39428__bvx%2Fio",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node5":{
"core":"c8n_1x3_shard1_replica_n3",
"base_url":"http://127.0.0.1:41305/_bvx/io",
"node_name":"127.0.0.1:41305__bvx%2Fio",
"state":"recovering",
"type":"NRT"},
"core_node6":{
"core":"c8n_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:35996/_bvx/io",
"node_name":"127.0.0.1:35996__bvx%2Fio",
"state":"down",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([8A506194BB9D6627:2045E4E15610BDF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:2005)
at org.apache.solr.cloud.HttpPartitionTest.testRf3(HttpPartitionTest.java:517)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitShardWithRule
Error Message:
Error from server at http://127.0.0.1:43837: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43837: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
at __randomizedtesting.SeedInfo.seed([8A506194BB9D6627:B5EB026266F3D0C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.api.collections.ShardSplitTest
Error Message:
43 threads leaked from SUITE scope at org.apache.solr.cloud.api.collections.ShardSplitTest: 1) Thread[id=24228, name=qtp1163358027-24228, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=24247, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=24261, name=qtp1353154897-24261, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=24268, name=Scheduler-267861909, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=24267, name=qtp1353154897-24267, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=24227, name=qtp1163358027-24227, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=24291, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=24235, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=24262, name=qtp1353154897-24262-acceptor-0@101d1f3a-ServerConnector@153475b9{HTTP/1.1,[http/1.1]}{127.0.0.1:35405}, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=24224, name=qtp1163358027-24224, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=24294, name=zkConnectionManagerCallback-7783-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=24249, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 13) Thread[id=24260, name=qtp1353154897-24260, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=24225, name=qtp1163358027-24225-acceptor-0@41235845-ServerConnector@ae84a53{HTTP/1.1,[http/1.1]}{127.0.0.1:37055}, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=24237, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=24226, name=qtp1163358027-24226, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=24229, name=qtp1163358027-24229, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=24274, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=24280, name=zkConnectionManagerCallback-7776-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=24255, name=searcherExecutor-5161-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=24263, name=qtp1353154897-24263, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=24273, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=24264, name=qtp1353154897-24264, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=24272, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=24292, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 26) Thread[id=24243, name=zkConnectionManagerCallback-7761-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=24236, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=24241, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 29) Thread[id=24289, name=Thread-5495, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 30) Thread[id=24248, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 31) Thread[id=24250, name=zkConnectionManagerCallback-7768-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=24266, name=qtp1353154897-24266, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=24251, name=Scheduler-1050442529, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=24223, name=qtp1163358027-24223, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=24259, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=24293, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 37) Thread[id=24230, name=qtp1163358027-24230, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=24231, name=Scheduler-162405083, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=24245, name=Thread-5486, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 40) Thread[id=24278, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 41) Thread[id=24265, name=qtp1353154897-24265, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 42) Thread[id=24242, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 43) Thread[id=24279, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 43 threads leaked from SUITE scope at org.apache.solr.cloud.api.collections.ShardSplitTest:
1) Thread[id=24228, name=qtp1163358027-24228, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=24247, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=24261, name=qtp1353154897-24261, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=24268, name=Scheduler-267861909, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=24267, name=qtp1353154897-24267, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=24227, name=qtp1163358027-24227, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=24291, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=24235, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=24262, name=qtp1353154897-24262-acceptor-0@101d1f3a-ServerConnector@153475b9{HTTP/1.1,[http/1.1]}{127.0.0.1:35405}, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=24224, name=qtp1163358027-24224, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=24294, name=zkConnectionManagerCallback-7783-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=24249, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
13) Thread[id=24260, name=qtp1353154897-24260, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=24225, name=qtp1163358027-24225-acceptor-0@41235845-ServerConnector@ae84a53{HTTP/1.1,[http/1.1]}{127.0.0.1:37055}, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=24237, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=24226, name=qtp1163358027-24226, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=24229, name=qtp1163358027-24229, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=24274, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
19) Thread[id=24280, name=zkConnectionManagerCallback-7776-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20) Thread[id=24255, name=searcherExecutor-5161-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21) Thread[id=24263, name=qtp1353154897-24263, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
22) Thread[id=24273, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
23) Thread[id=24264, name=qtp1353154897-24264, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
24) Thread[id=24272, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
25) Thread[id=24292, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
26) Thread[id=24243, name=zkConnectionManagerCallback-7761-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
27) Thread[id=24236, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
28) Thread[id=24241, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
29) Thread[id=24289, name=Thread-5495, state=WAITING, group=TGRP-ShardSplitTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)
30) Thread[id=24248, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
31) Thread[id=24250, name=zkConnectionManagerCallback-7768-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
32) Thread[id=24266, name=qtp1353154897-24266, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
33) Thread[id=24251, name=Scheduler-1050442529, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
34) Thread[id=24223, name=qtp1163358027-24223, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
35) Thread[id=24259, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
36) Thread[id=24293, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
37) Thread[id=24230, name=qtp1163358027-24230, state=RUNNABLE, group=TGRP-ShardSplitTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
38) Thread[id=24231, name=Scheduler-162405083, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
39) Thread[id=24245, name=Thread-5486, state=WAITING, group=TGRP-ShardSplitTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)
40) Thread[id=24278, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
41) Thread[id=24265, name=qtp1353154897-24265, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
42) Thread[id=24242, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
43) Thread[id=24279, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
at __randomizedtesting.SeedInfo.seed([8A506194BB9D6627]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.api.collections.ShardSplitTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=24280, name=zkConnectionManagerCallback-7776-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=24255, name=searcherExecutor-5161-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=24261, name=qtp1353154897-24261, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=24292, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 5) Thread[id=24268, name=Scheduler-267861909, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=24243, name=zkConnectionManagerCallback-7761-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=24241, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 8) Thread[id=24289, name=Thread-5495, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 9) Thread[id=24248, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:37002), state=TIMED_WAITING, group=TGRP-ShardSplitTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 10) Thread[id=24224, name=qtp1163358027-24224, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.IOUtil.drain(Native Method) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=24294, name=zkConnectionManagerCallback-7783-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=24250, name=zkConnectionManagerCallback-7768-thread-1, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=24251, name=Scheduler-1050442529, state=WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=24260, name=qtp1353154897-24260, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:105) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:401) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$251/1967084154.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=24230, name=qtp1163358027-24230, state=RUNNABLE, group=TGRP-ShardSplitTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:105) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:401) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=24231, name=Scheduler-162405083, state=TIMED_WAITING, group=TGRP-ShardSplitTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=24245, name=Thread-5486, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 18) Thread[id=24278, name=TEST-ShardSplitTest.testSplitShardWithRule-seed#[8A506194BB9D6627]-SendThread(127.0.0.1:370
[...truncated too long message...]
dApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2073462754
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2073462754
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null2073462754
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1204152221
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1204152221
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1204152221
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 73ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 268 minutes 30 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 63 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/63/
6 tests failed.
FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef
Error Message:
ReaderPool is already closed
Stack Trace:
org.apache.lucene.store.AlreadyClosedException: ReaderPool is already closed
at __randomizedtesting.SeedInfo.seed([403003F3F547ABF5:A9AD74C1838E4C08]:0)
at org.apache.lucene.index.ReaderPool.get(ReaderPool.java:345)
at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3302)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514)
at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:396)
at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:332)
at org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:465)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ReplicationFactorTest.test
Error Message:
Didn't see all replicas for shard shard1 in repfacttest_c8n_1x3 come up within 30000 ms! ClusterState: { "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node2":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:39733", "node_name":"127.0.0.1:39733_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "repfacttest_c8n_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node3":{ "core":"repfacttest_c8n_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:39568", "node_name":"127.0.0.1:39568_", "state":"active", "type":"NRT"}, "core_node5":{ "core":"repfacttest_c8n_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:44729", "node_name":"127.0.0.1:44729_", "state":"down", "type":"NRT"}, "core_node6":{ "core":"repfacttest_c8n_1x3_shard1_replica_n4", "base_url":"http://127.0.0.1:39733", "node_name":"127.0.0.1:39733_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{"core_node66":{ "core":"collection1_shard1_replica_n65", "base_url":"http://127.0.0.1:36679", "node_name":"127.0.0.1:36679_", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{"core_node62":{ "core":"collection1_shard2_replica_n61", "base_url":"http://127.0.0.1:39568", "node_name":"127.0.0.1:39568_", "state":"active", "type":"NRT", "leader":"true"}}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{"core_node64":{ "core":"collection1_shard3_replica_n63", "base_url":"http://127.0.0.1:44729", "node_name":"127.0.0.1:44729_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in repfacttest_c8n_1x3 come up within 30000 ms! ClusterState: {
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:39733",
"node_name":"127.0.0.1:39733_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"repfacttest_c8n_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node3":{
"core":"repfacttest_c8n_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:39568",
"node_name":"127.0.0.1:39568_",
"state":"active",
"type":"NRT"},
"core_node5":{
"core":"repfacttest_c8n_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:44729",
"node_name":"127.0.0.1:44729_",
"state":"down",
"type":"NRT"},
"core_node6":{
"core":"repfacttest_c8n_1x3_shard1_replica_n4",
"base_url":"http://127.0.0.1:39733",
"node_name":"127.0.0.1:39733_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-d554ffff",
"state":"active",
"replicas":{"core_node66":{
"core":"collection1_shard1_replica_n65",
"base_url":"http://127.0.0.1:36679",
"node_name":"127.0.0.1:36679_",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"d5550000-2aa9ffff",
"state":"active",
"replicas":{"core_node62":{
"core":"collection1_shard2_replica_n61",
"base_url":"http://127.0.0.1:39568",
"node_name":"127.0.0.1:39568_",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard3":{
"range":"2aaa0000-7fffffff",
"state":"active",
"replicas":{"core_node64":{
"core":"collection1_shard3_replica_n63",
"base_url":"http://127.0.0.1:44729",
"node_name":"127.0.0.1:44729_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([62E113F453A6CEC6:EAB52C2EFD5AA33E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:2005)
at org.apache.solr.cloud.ReplicationFactorTest.testRf3(ReplicationFactorTest.java:376)
at org.apache.solr.cloud.ReplicationFactorTest.test(ReplicationFactorTest.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest.testListenerAcceptance
Error Message:
expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([62E113F453A6CEC6:73554913EA2FDA30]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.NodeLostTriggerTest.testListenerAcceptance(NodeLostTriggerTest.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
there should be new MOVERPLICA ops
Stack Trace:
java.lang.AssertionError: there should be new MOVERPLICA ops
at __randomizedtesting.SeedInfo.seed([62E113F453A6CEC6:C91B0EE18C7A48E8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
Error Message:
should be at least one inactive event
Stack Trace:
java.lang.AssertionError: should be at least one inactive event
at __randomizedtesting.SeedInfo.seed([62E113F453A6CEC6:7FCDD38632E5E9CD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([62E113F453A6CEC6:12A2576CA69BDEB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger(SearchRateTriggerTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 1315 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexFileDeleter -Dtests.method=testExcInDecRef -Dtests.seed=403003F3F547ABF5 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-GT -Dtests.timezone=America/Curacao -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 1.72s J1 | TestIndexFileDeleter.testExcInDecRef <<<
[junit4] > Throwable #1: org.apache.lucene.store.AlreadyClosedException: ReaderPool is already closed
[junit4] > at __randomizedtesting.SeedInfo.seed([403003F3F547ABF5:A9AD74C1838E4C08]:0)
[junit4] > at org.apache.lucene.index.ReaderPool.get(ReaderPool.java:345)
[junit4] > at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3302)
[junit4] > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:514)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:396)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:332)
[junit4] > at org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:465)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/build/core/test/J1/temp/lucene.index.TestIndexFileDeleter_403003F3F547ABF5-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {field=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=Lucene50(blocksize=128), content=FST50}, docValues:{}, maxPointsInLeafNode=360, maxMBSortInHeap=6.275982034690589, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7de9511f), locale=es-GT, timezone=America/Curacao
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=263398584,total=385351680
[junit4] 2> NOTE: All tests run in this JVM: [TestCustomSearcherSort, TestOrdinalMap, TestRecyclingByteBlockAllocator, TestPrefixQuery, TestDocumentsWriterDeleteQueue, TestDirectMonotonic, TestByteSlices, TestToken, TestStressNRT, TestForTooMuchCloning, TestRadixSelector, TestLongValuesSource, TestSearchForDuplicates, TestSloppyMath, TestOmitTf, TestUpgradeIndexMergePolicy, TestReaderWrapperDVTypeCheck, TestNGramPhraseQuery, TestLucene50CompoundFormat, TestLockFactory, TestSwappedIndexFiles, TestFastDecompressionMode, TestPerFieldDocValuesFormat, TestCharFilter, TestBooleanQueryVisitSubscorers, TestIntRangeFieldQueries, TestSingleInstanceLockFactory, TestTransactions, TestParallelReaderEmptyIndex, TestBytesRef, TestTimSorterWorstCase, TestMmapDirectory, TestAtomicUpdate, TestMultiTermConstantScore, TestSameScoresWithThreads, TestWeakIdentityMap, Test2BTerms, TestBlockPostingsFormat2, TestLogMergePolicy, TestLevenshteinAutomata, TestOmitPositions, Test2BBKDPoints, TestIntArrayDocIdSet, TestReusableStringReader, TestCrash, TestDocValuesFieldUpdates, TestLucene50StoredFieldsFormatHighCompression, TestPrefixCodedTerms, TestNot, TestSpanContainQuery, TestMultiset, Test2BPoints, TestReqOptSumScorer, TestBytesRefArray, TestMinShouldMatch2, TestAxiomaticF2LOG, TestSearch, TestMergeRateLimiter, TestMinimize, TestSimilarityProvider, TestFutureArrays, TestDoc, TestMergeSchedulerExternal, TestExternalCodecs, TestDocValuesRewriteMethod, TestPointQueries, TestCustomNorms, TestFixedBitDocIdSet, TestBooleanQuery, TestGeoUtils, TestSimilarity, TestNRTReaderCleanup, TestMultiDocValues, TestPrefixInBooleanQuery, TestRegexpRandom2, TestIndexWriterLockRelease, TestNoMergePolicy, TestAxiomaticF1EXP, TestAutomaton, TestApproximationSearchEquivalence, TestWildcard, TestTermsEnum2, TestFSTs, TestGraphTokenizers, TestAddIndexes, TestShardSearching, TestMultiMMap, TestSearcherManager, TestDeletionPolicy, TestBooleanOr, TestBasics, TestSloppyPhraseQuery, TestIndexWriterDelete, TestNRTThreads, TestDuelingCodecs, TestIndexWriterCommit, TestDirectoryReaderReopen, TestTimeLimitingCollector, TestSearchAfter, TestConcurrentMergeScheduler, TestHighCompressionMode, TestDocumentsWriterStallControl, TestStressIndexing, TestSnapshotDeletionPolicy, TestIndexWriterUnicode, TestIndexWriterForceMerge, TestArrayUtil, TestDirectoryReader, TestPhraseQuery, TestComplexExplanations, TestSegmentReader, TestParallelCompositeReader, TestIndexWriterNRTIsCurrent, TestMixedCodecs, TestTransactionRollback, TestSpanExplanations, TestLazyProxSkipping, TestDeterminism, TestBagOfPostings, TestThreadedForceMerge, TestForceMergeForever, TestDocumentWriter, TestCrashCausesCorruptIndex, TestFuzzyQuery, TestSimilarity2, TestTermVectorsReader, TestUnicodeUtil, TestTermScorer, TestSumDocFreq, TestIndexableField, TestWildcardRandom, TestDirectory, TestRegexpRandom, TestPayloadsOnVectors, TestSpanMultiTermQueryWrapper, TestIndexFileDeleter]
[junit4] Completed [323/495 (1!)] on J1 in 4.16s, 10 tests, 1 error <<< FAILURES!
[...truncated 11256 lines...]
[junit4] Suite: org.apache.solr.cloud.ReplicationFactorTest
[junit4] 2> 668550 INFO (SUITE-ReplicationFactorTest-seed#[62E113F453A6CEC6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/init-core-data-001
[junit4] 2> 668550 WARN (SUITE-ReplicationFactorTest-seed#[62E113F453A6CEC6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=38 numCloses=38
[junit4] 2> 668550 INFO (SUITE-ReplicationFactorTest-seed#[62E113F453A6CEC6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 668601 INFO (SUITE-ReplicationFactorTest-seed#[62E113F453A6CEC6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 668601 INFO (SUITE-ReplicationFactorTest-seed#[62E113F453A6CEC6]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 668673 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 668675 INFO (Thread-285) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 668675 INFO (Thread-285) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 668694 ERROR (Thread-285) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 668783 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ZkTestServer start zk server on port:38383
[junit4] 2> 669059 INFO (zkConnectionManagerCallback-390-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 669206 INFO (zkConnectionManagerCallback-392-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 669357 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 669358 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 669360 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 669394 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 669395 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 669396 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 669396 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 669414 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 669415 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 669416 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 669417 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 669484 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 670900 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 670950 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 670951 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 670951 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 670972 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60e15283{/,null,AVAILABLE}
[junit4] 2> 671081 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e966537{HTTP/1.1,[http/1.1]}{127.0.0.1:36296}
[junit4] 2> 671081 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server Started @671278ms
[junit4] 2> 671081 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=39733, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/control-001/cores}
[junit4] 2> 671082 ERROR (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 671082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 671082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 671082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 671082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 671082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-24T19:32:10.899Z
[junit4] 2> 671155 INFO (zkConnectionManagerCallback-394-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 671191 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 671191 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/control-001/solr.xml
[junit4] 2> 671207 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 671207 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 671208 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 671245 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38383/solr
[junit4] 2> 671313 INFO (zkConnectionManagerCallback-398-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 671361 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10237e59d160003, likely client has closed socket
[junit4] 2> 671438 INFO (zkConnectionManagerCallback-400-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 672615 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 672616 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39733_
[junit4] 2> 672630 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.Overseer Overseer (id=72682003313852420-127.0.0.1:39733_-n_0000000000) starting
[junit4] 2> 672957 INFO (zkConnectionManagerCallback-407-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 672992 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38383/solr ready
[junit4] 2> 673008 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39733_
[junit4] 2> 673011 INFO (OverseerStateUpdate-72682003313852420-127.0.0.1:39733_-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 673064 INFO (zkCallback-406-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 674983 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 675164 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 675165 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 675166 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39733_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/control-001/cores
[junit4] 2> 675435 INFO (zkConnectionManagerCallback-412-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 675477 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 675478 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38383/solr ready
[junit4] 2> 675479 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=59112,localport=39733], receiveBufferSize:531000
[junit4] 2> 675526 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36296,localport=39100], receiveBufferSize=530904
[junit4] 2> 675599 INFO (qtp633993717-1593) [n:127.0.0.1:39733_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39733_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 675712 INFO (OverseerThreadFactory-550-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 676099 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=59116,localport=39733], receiveBufferSize:531000
[junit4] 2> 676102 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36296,localport=39104], receiveBufferSize=530904
[junit4] 2> 676102 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 676103 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 677426 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 677480 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 678782 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 678943 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 678944 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 678944 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 678997 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 679035 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5729639729189401]
[junit4] 2> 679065 WARN (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 679555 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 679555 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 679556 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 679556 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 679557 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=21.5224609375, floorSegmentMB=0.708984375, forceMergeDeletesPctAllowed=2.2546097080154603, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 679571 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d9516c8[control_collection_shard1_replica_n1] main]
[junit4] 2> 679591 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 679592 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 679593 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 679594 INFO (searcherExecutor-553-thread-1-processing-n:127.0.0.1:39733_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@2d9516c8[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 679594 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601375137338228736
[junit4] 2> 679683 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39733/control_collection_shard1_replica_n1/
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39733/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 679740 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 679756 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39733/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 679864 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 679866 INFO (qtp633993717-1588) [n:127.0.0.1:39733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3764
[junit4] 2> 679939 INFO (qtp633993717-1593) [n:127.0.0.1:39733_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 680067 INFO (zkCallback-399-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 680945 INFO (qtp633993717-1593) [n:127.0.0.1:39733_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39733_&wt=javabin&version=2} status=0 QTime=5346
[junit4] 2> 681051 INFO (zkConnectionManagerCallback-417-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 681064 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 681065 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38383/solr ready
[junit4] 2> 681065 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 681066 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=59136,localport=39733], receiveBufferSize:531000
[junit4] 2> 681087 INFO (SocketProxy-Acceptor-39733) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36296,localport=39124], receiveBufferSize=530904
[junit4] 2> 681137 INFO (qtp633993717-1589) [n:127.0.0.1:39733_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 681158 INFO (OverseerThreadFactory-550-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 681158 INFO (OverseerCollectionConfigSetProcessor-72682003313852420-127.0.0.1:39733_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 681159 WARN (OverseerThreadFactory-550-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 681539 INFO (qtp633993717-1589) [n:127.0.0.1:39733_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 681559 INFO (qtp633993717-1589) [n:127.0.0.1:39733_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=421
[junit4] 2> 682342 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001 of type NRT
[junit4] 2> 682343 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 682451 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 682451 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 682452 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 682452 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54313f37{/,null,AVAILABLE}
[junit4] 2> 682452 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2004ffb8{HTTP/1.1,[http/1.1]}{127.0.0.1:46411}
[junit4] 2> 682452 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server Started @682649ms
[junit4] 2> 682452 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=39568, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001/cores}
[junit4] 2> 682453 ERROR (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 682453 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 682453 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 682453 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 682453 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 682453 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-24T19:32:22.270Z
[junit4] 2> 682511 INFO (zkConnectionManagerCallback-419-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 682533 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 682533 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001/solr.xml
[junit4] 2> 682537 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 682537 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 682538 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 682682 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38383/solr
[junit4] 2> 682722 INFO (zkConnectionManagerCallback-423-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 682897 INFO (zkConnectionManagerCallback-425-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 682967 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 682968 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 683043 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 683043 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39568_
[junit4] 2> 683071 INFO (zkCallback-399-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 683088 INFO (zkCallback-406-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 683206 INFO (zkCallback-416-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 683329 INFO (zkCallback-424-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 683365 INFO (OverseerCollectionConfigSetProcessor-72682003313852420-127.0.0.1:39733_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 684009 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 684083 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 684083 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 684117 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001/cores
[junit4] 2> 684210 INFO (zkConnectionManagerCallback-432-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 684223 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 684224 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:39568_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38383/solr ready
[junit4] 2> 684503 INFO (SocketProxy-Acceptor-39568) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55380,localport=39568], receiveBufferSize:531000
[junit4] 2> 684519 INFO (SocketProxy-Acceptor-39568) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46411,localport=36460], receiveBufferSize=530904
[junit4] 2> 684943 INFO (qtp1328946337-1653) [n:127.0.0.1:39568_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:39568_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 685398 INFO (OverseerThreadFactory-550-thread-3) [ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:39568_ for creating new replica
[junit4] 2> 685440 INFO (SocketProxy-Acceptor-39568) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55388,localport=39568], receiveBufferSize:531000
[junit4] 2> 685474 INFO (SocketProxy-Acceptor-39568) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46411,localport=36468], receiveBufferSize=530904
[junit4] 2> 685614 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ x:collection1_shard2_replica_n61] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 685983 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 686130 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 687055 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 687186 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 687220 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 687220 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 687220 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
[junit4] 2> 687222 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5729639729189401]
[junit4] 2> 687274 WARN (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 688111 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 688111 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 688112 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 688112 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 688113 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=21.5224609375, floorSegmentMB=0.708984375, forceMergeDeletesPctAllowed=2.2546097080154603, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 688126 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@32dd78ec[collection1_shard2_replica_n61] main]
[junit4] 2> 688127 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 688128 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 688128 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 688180 INFO (searcherExecutor-564-thread-1-processing-n:127.0.0.1:39568_ x:collection1_shard2_replica_n61 c:collection1 s:shard2) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@32dd78ec[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 688180 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601375146341302272
[junit4] 2> 688255 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node62=0}, version=0}
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39568/collection1_shard2_replica_n61/
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:39568/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 688348 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 688385 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39568/collection1_shard2_replica_n61/ shard2
[junit4] 2> 688456 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 688477 INFO (qtp1328946337-1648) [n:127.0.0.1:39568_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2863
[junit4] 2> 688529 INFO (qtp1328946337-1653) [n:127.0.0.1:39568_ c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39568_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=3586
[junit4] 2> 688530 INFO (OverseerCollectionConfigSetProcessor-72682003313852420-127.0.0.1:39733_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 688566 INFO (zkCallback-424-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 689774 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-2-001 of type NRT
[junit4] 2> 689775 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 689862 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 689863 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 689863 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 689883 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@213c82ee{/,null,AVAILABLE}
[junit4] 2> 689883 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4efe6b8f{HTTP/1.1,[http/1.1]}{127.0.0.1:33435}
[junit4] 2> 689883 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.e.j.s.Server Started @690080ms
[junit4] 2> 689883 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=44729, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-2-001/cores}
[junit4] 2> 689884 ERROR (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 689884 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 689919 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 689920 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 689920 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 689920 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-24T19:32:29.737Z
[junit4] 2> 690067 INFO (zkConnectionManagerCallback-434-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 690082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 690082 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.ReplicationFactorTest_62E113F453A6CEC6-001/shard-2-001/solr.xml
[junit4] 2> 690086 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 690118 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 690120 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 690177 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38383/solr
[junit4] 2> 690230 INFO (zkConnectionManagerCallback-438-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 690404 INFO (zkConnectionManagerCallback-440-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 690500 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 690556 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 690607 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 690607 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44729_
[junit4] 2> 690628 INFO (zkCallback-399-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 690933 INFO (zkCallback-431-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 690940 INFO (zkCallback-424-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 690944 INFO (zkCallback-406-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 690944 INFO (zkCallback-416-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 691038 INFO (zkCallback-439-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 693241 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 693348 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14ef063
[junit4] 2> 693348 INFO (TEST-ReplicationFactorTest.test-seed#[62E113F453A6CEC6]) [n:127.0.0.1:44729_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jet
[...truncated too long message...]
-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null921515627
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null921515627
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null921515627
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1617709717
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1617709717
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1617709717
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 52ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 320 minutes 49 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 62 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/62/
4 tests failed.
FAILED: org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete
Error Message:
Error from server at https://127.0.0.1:48926/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:48926/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([951DFCBA5992E48A:36E7521FDE7A0E2F]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48926/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:202)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
events: [CapturedEvent{timestamp=27448637635779656, stage=STARTED, actionName='null', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637873439256, stage=BEFORE_ACTION, actionName='compute_plan', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637943265756, stage=AFTER_ACTION, actionName='compute_plan', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637957973706, stage=BEFORE_ACTION, actionName='execute_plan', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448646035340006, stage=AFTER_ACTION, actionName='execute_plan', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/618462ff9111b6T5glisjblhfl5vgaq5b3334guy/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/618462ff9111b6T5glisjblhfl5vgaq5b3334guy/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448646059888856, stage=SUCCEEDED, actionName='null', event={ "id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy", "source":"index_size_trigger2", "eventTime":27448633470292406, "eventType":"INDEXSIZE", "properties":{ "__start__":1, "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448637628418156, "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655573912056, stage=STARTED, actionName='null', event={ "id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1", "source":"index_size_trigger2", "eventTime":27448651452436056, "eventType":"INDEXSIZE", "properties":{ "__start__":3, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448655564496306, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655748292506, stage=BEFORE_ACTION, actionName='compute_plan', event={ "id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1", "source":"index_size_trigger2", "eventTime":27448651452436056, "eventType":"INDEXSIZE", "properties":{ "__start__":3, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448655564496306, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655834285806, stage=AFTER_ACTION, actionName='compute_plan', event={ "id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1", "source":"index_size_trigger2", "eventTime":27448651452436056, "eventType":"INDEXSIZE", "properties":{ "__start__":3, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448655564496306, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655860850756, stage=BEFORE_ACTION, actionName='execute_plan', event={ "id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1", "source":"index_size_trigger2", "eventTime":27448651452436056, "eventType":"INDEXSIZE", "properties":{ "__start__":3, "aboveSize":{"testSplitIntegration_collection":[ "{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}", "{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]}, "belowSize":{}, "_enqueue_time_":27448655564496306, "requestedOps":[ "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}", "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={ "trigger":"index_size_trigger2", "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED"], "afterAction":[ "compute_plan", "execute_plan"], "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener", "beforeAction":[ "compute_plan", "execute_plan"]}, message='null'}] expected:<6> but was:<10>
Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=27448637635779656, stage=STARTED, actionName='null', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637873439256, stage=BEFORE_ACTION, actionName='compute_plan', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637943265756, stage=AFTER_ACTION, actionName='compute_plan', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448637957973706, stage=BEFORE_ACTION, actionName='execute_plan', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448646035340006, stage=AFTER_ACTION, actionName='execute_plan', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/618462ff9111b6T5glisjblhfl5vgaq5b3334guy/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/618462ff9111b6T5glisjblhfl5vgaq5b3334guy/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448646059888856, stage=SUCCEEDED, actionName='null', event={
"id":"618462ff9111b6T5glisjblhfl5vgaq5b3334guy",
"source":"index_size_trigger2",
"eventTime":27448633470292406,
"eventType":"INDEXSIZE",
"properties":{
"__start__":1,
"aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448637628418156,
"requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655573912056, stage=STARTED, actionName='null', event={
"id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1",
"source":"index_size_trigger2",
"eventTime":27448651452436056,
"eventType":"INDEXSIZE",
"properties":{
"__start__":3,
"aboveSize":{"testSplitIntegration_collection":[
"{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448655564496306,
"requestedOps":[
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655748292506, stage=BEFORE_ACTION, actionName='compute_plan', event={
"id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1",
"source":"index_size_trigger2",
"eventTime":27448651452436056,
"eventType":"INDEXSIZE",
"properties":{
"__start__":3,
"aboveSize":{"testSplitIntegration_collection":[
"{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448655564496306,
"requestedOps":[
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655834285806, stage=AFTER_ACTION, actionName='compute_plan', event={
"id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1",
"source":"index_size_trigger2",
"eventTime":27448651452436056,
"eventType":"INDEXSIZE",
"properties":{
"__start__":3,
"aboveSize":{"testSplitIntegration_collection":[
"{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448655564496306,
"requestedOps":[
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}, CapturedEvent{timestamp=27448655860850756, stage=BEFORE_ACTION, actionName='execute_plan', event={
"id":"6184672f62ce58T5glisjblhfl5vgaq5b3334gv1",
"source":"index_size_trigger2",
"eventTime":27448651452436056,
"eventType":"INDEXSIZE",
"properties":{
"__start__":3,
"aboveSize":{"testSplitIntegration_collection":[
"{\"core_node10\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":12,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":12,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"base_url\":\"http://127.0.0.1:10007/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":12,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n \"shard\":\"shard1_1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}",
"{\"core_node7\":{\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":13,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":13,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10006_solr\",\n \"base_url\":\"http://127.0.0.1:10006/solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":13,\n \"__bytes__\":1,\n \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n \"shard\":\"shard1_0\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
"belowSize":{},
"_enqueue_time_":27448655564496306,
"requestedOps":[
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_1\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}",
"Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
"trigger":"index_size_trigger2",
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED"],
"afterAction":[
"compute_plan",
"execute_plan"],
"class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
"beforeAction":[
"compute_plan",
"execute_plan"]}, message='null'}] expected:<6> but was:<10>
at __randomizedtesting.SeedInfo.seed([951DFCBA5992E48A:AC9345FA766D2D74]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:314)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
Error Message:
should be at least one inactive event
Stack Trace:
java.lang.AssertionError: should be at least one inactive event
at __randomizedtesting.SeedInfo.seed([951DFCBA5992E48A:88313CC838D1C381]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([951DFCBA5992E48A:F6D6CA38C05D97A7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger(SearchRateTriggerTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13377 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/init-core-data-001
[junit4] 2> 1630469 WARN (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
[junit4] 2> 1630469 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1630471 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1630472 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1630620 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/tempDir-001
[junit4] 2> 1630620 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1630649 INFO (Thread-2827) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1630649 INFO (Thread-2827) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1630716 ERROR (Thread-2827) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1630749 INFO (SUITE-SearchRateTriggerTest-seed#[951DFCBA5992E48A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:39646
[junit4] 2> 1630778 INFO (zkConnectionManagerCallback-3085-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1630790 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1630804 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1630805 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1630807 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1630809 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1630809 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1630809 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1630810 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d42d72e{/solr,null,AVAILABLE}
[junit4] 2> 1630810 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7de18096{SSL,[ssl, http/1.1]}{127.0.0.1:57862}
[junit4] 2> 1630810 INFO (jetty-launcher-3082-thread-1) [ ] o.e.j.s.Server Started @1631108ms
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57862}
[junit4] 2> 1630811 ERROR (jetty-launcher-3082-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1630811 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-23T15:58:31.226Z
[junit4] 2> 1630882 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1630882 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1630882 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1630882 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cbc796e{/solr,null,AVAILABLE}
[junit4] 2> 1630883 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7cfc41d5{SSL,[ssl, http/1.1]}{127.0.0.1:50570}
[junit4] 2> 1630883 INFO (jetty-launcher-3082-thread-3) [ ] o.e.j.s.Server Started @1631180ms
[junit4] 2> 1630883 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50570}
[junit4] 2> 1630883 ERROR (jetty-launcher-3082-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1630883 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1630884 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1630884 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1630884 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1630884 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-23T15:58:31.299Z
[junit4] 2> 1630925 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1630925 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1630926 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1630931 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@160872ca{/solr,null,AVAILABLE}
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@87f835{SSL,[ssl, http/1.1]}{127.0.0.1:57882}
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.e.j.s.Server Started @1631229ms
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57882}
[junit4] 2> 1630932 ERROR (jetty-launcher-3082-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1630932 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-23T15:58:31.347Z
[junit4] 2> 1630948 INFO (zkConnectionManagerCallback-3087-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1630949 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1630975 INFO (zkConnectionManagerCallback-3089-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1630976 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1630994 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1630994 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1630994 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1630997 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a027873{/solr,null,AVAILABLE}
[junit4] 2> 1630997 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d3c56eb{SSL,[ssl, http/1.1]}{127.0.0.1:54333}
[junit4] 2> 1630997 INFO (jetty-launcher-3082-thread-4) [ ] o.e.j.s.Server Started @1631294ms
[junit4] 2> 1630997 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54333}
[junit4] 2> 1630998 ERROR (jetty-launcher-3082-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1630998 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1630998 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1630998 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1630998 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1630998 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-23T15:58:31.413Z
[junit4] 2> 1631009 INFO (zkConnectionManagerCallback-3091-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631009 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1631015 INFO (zkConnectionManagerCallback-3093-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631016 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1631025 INFO (jetty-launcher-3082-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39646/solr
[junit4] 2> 1631029 INFO (jetty-launcher-3082-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39646/solr
[junit4] 2> 1631031 INFO (zkConnectionManagerCallback-3098-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631034 INFO (jetty-launcher-3082-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39646/solr
[junit4] 2> 1631035 INFO (zkConnectionManagerCallback-3102-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631037 INFO (jetty-launcher-3082-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39646/solr
[junit4] 2> 1631039 INFO (zkConnectionManagerCallback-3106-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631042 INFO (zkConnectionManagerCallback-3109-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631044 INFO (zkConnectionManagerCallback-3111-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631121 INFO (zkConnectionManagerCallback-3113-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631132 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1631133 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50570_solr
[junit4] 2> 1631134 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.Overseer Overseer (id=73849545419849738-127.0.0.1:50570_solr-n_0000000000) starting
[junit4] 2> 1631146 INFO (zkConnectionManagerCallback-3115-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631157 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1631158 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54333_solr
[junit4] 2> 1631176 INFO (zkConnectionManagerCallback-3117-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631183 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1631186 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1631189 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1631189 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57882_solr
[junit4] 2> 1631264 INFO (zkCallback-3114-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1631264 INFO (zkCallback-3112-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1631290 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1631293 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1631336 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1631336 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57862_solr
[junit4] 2> 1631352 INFO (zkCallback-3116-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1631361 INFO (zkCallback-3112-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1631361 INFO (zkCallback-3116-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1631388 INFO (zkConnectionManagerCallback-3130-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631393 INFO (zkCallback-3114-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1631524 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1631525 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39646/solr ready
[junit4] 2> 1631532 INFO (zkCallback-3110-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1631562 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1631562 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50570_solr
[junit4] 2> 1631563 INFO (zkCallback-3116-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1631563 INFO (zkCallback-3110-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1631563 INFO (zkCallback-3114-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1631568 INFO (zkCallback-3112-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1631616 INFO (zkCallback-3129-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1632439 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50570.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632480 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50570.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632480 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50570.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632482 INFO (jetty-launcher-3082-thread-3) [n:127.0.0.1:50570_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/tempDir-001/node3/.
[junit4] 2> 1632702 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57882.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632719 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57882.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632719 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57882.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632739 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57862.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632743 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54333.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632812 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/tempDir-001/node2/.
[junit4] 2> 1632820 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54333.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632820 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54333.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632822 INFO (jetty-launcher-3082-thread-4) [n:127.0.0.1:54333_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/tempDir-001/node4/.
[junit4] 2> 1632849 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57862.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632849 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57862.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@f1fcf93
[junit4] 2> 1632850 INFO (jetty-launcher-3082-thread-1) [n:127.0.0.1:57862_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerTest_951DFCBA5992E48A-001/tempDir-001/node1/.
[junit4] 2> 1632884 INFO (zkConnectionManagerCallback-3135-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1632885 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1632886 INFO (jetty-launcher-3082-thread-2) [n:127.0.0.1:57882_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39646/solr ready
[junit4] 2> 1632976 INFO (zkConnectionManagerCallback-3140-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1632978 INFO (jetty-launcher-3082-thread-4) [n:12
[...truncated too long message...]
file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null716614312
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null716614312
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null716614312
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1438462582
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1438462582
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1438462582
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 47ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 172 minutes 15 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 61 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/61/
4 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([71E476A51D330AAC]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([71E476A51D330AAC]:0)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
waitFor not elapsed but produced an event
Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
at __randomizedtesting.SeedInfo.seed([71E476A51D330AAC:122F402784FC7981]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([71E476A51D330AAC:2CAC682CD2F5ACE3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:579)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13397 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> 1671930 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_71E476A51D330AAC-001/init-core-data-001
[junit4] 2> 1671964 WARN (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=11 numCloses=11
[junit4] 2> 1671964 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1671966 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1671966 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_71E476A51D330AAC-001/tempDir-001
[junit4] 2> 1671967 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1671985 INFO (Thread-5188) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1671985 INFO (Thread-5188) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1672025 ERROR (Thread-5188) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1672186 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37251
[junit4] 2> 1672218 INFO (zkConnectionManagerCallback-5650-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672228 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1672228 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1672324 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1672324 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1672324 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1672329 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3cb60c43{/solr,null,AVAILABLE}
[junit4] 2> 1672330 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6ac1ec6c{HTTP/1.1,[http/1.1]}{127.0.0.1:48588}
[junit4] 2> 1672330 INFO (jetty-launcher-5647-thread-1) [ ] o.e.j.s.Server Started @1672633ms
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48588}
[junit4] 2> 1672331 ERROR (jetty-launcher-5647-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1672331 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-22T02:59:50.505Z
[junit4] 2> 1672409 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1672409 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1672409 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1672418 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58215dd7{/solr,null,AVAILABLE}
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4fb69dd5{HTTP/1.1,[http/1.1]}{127.0.0.1:49685}
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.e.j.s.Server Started @1672721ms
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49685}
[junit4] 2> 1672419 ERROR (jetty-launcher-5647-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1672419 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-22T02:59:50.593Z
[junit4] 2> 1672485 INFO (zkConnectionManagerCallback-5652-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672488 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1672508 INFO (zkConnectionManagerCallback-5654-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672510 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1672538 INFO (jetty-launcher-5647-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37251/solr
[junit4] 2> 1672587 INFO (jetty-launcher-5647-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37251/solr
[junit4] 2> 1672596 INFO (zkConnectionManagerCallback-5658-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672596 INFO (zkConnectionManagerCallback-5662-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672599 INFO (zkConnectionManagerCallback-5664-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1672601 INFO (zkConnectionManagerCallback-5666-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1673623 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1673624 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49685_solr
[junit4] 2> 1673626 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.Overseer Overseer (id=73840821206515718-127.0.0.1:49685_solr-n_0000000000) starting
[junit4] 2> 1673638 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1673639 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48588_solr
[junit4] 2> 1673745 INFO (zkCallback-5663-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1673760 INFO (zkCallback-5665-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1673764 INFO (zkConnectionManagerCallback-5675-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1673772 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1673775 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37251/solr ready
[junit4] 2> 1673840 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1673842 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1673842 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49685_solr
[junit4] 2> 1673843 INFO (zkCallback-5663-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1673885 INFO (OverseerStateUpdate-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1673909 INFO (zkCallback-5674-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1673941 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1673942 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1673942 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1673984 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:48588_solr, 127.0.0.1:49685_solr]
[junit4] 2> 1673984 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1673984 DEBUG (OverseerAutoScalingTriggerThread-73840821206515718-127.0.0.1:49685_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1673987 DEBUG (ScheduledTrigger-4751-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1675039 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48588.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675049 DEBUG (ScheduledTrigger-4751-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1675060 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48588.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675060 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48588.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675062 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_71E476A51D330AAC-001/tempDir-001/node2/.
[junit4] 2> 1675129 INFO (zkConnectionManagerCallback-5680-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1675130 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1675131 INFO (jetty-launcher-5647-thread-1) [n:127.0.0.1:48588_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37251/solr ready
[junit4] 2> 1675289 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49685.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675379 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49685.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675379 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49685.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@689ba811
[junit4] 2> 1675421 INFO (jetty-launcher-5647-thread-2) [n:127.0.0.1:49685_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_71E476A51D330AAC-001/tempDir-001/node1/.
[junit4] 2> 1675499 INFO (zkConnectionManagerCallback-5682-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1675548 INFO (zkConnectionManagerCallback-5687-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1675549 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1675550 INFO (SUITE-IndexSizeTriggerTest-seed#[71E476A51D330AAC]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37251/solr ready
[junit4] 2> 1676042 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1676043 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1676043 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1676043 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1676048 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1676049 DEBUG (ScheduledTrigger-4751-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676059 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1676059 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1676062 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676083 DEBUG (ScheduledTrigger-4760-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676104 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676129 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676150 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676170 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676193 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676225 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676265 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676305 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676369 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676393 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676413 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676433 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676453 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676474 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676494 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676514 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676534 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676554 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676741 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676761 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676785 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676818 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[71E476A51D330AAC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
[junit4] 2> 1676829 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676894 DEBUG (simCloudManagerPool-4759-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testTrigger_collection"}}
[junit4] 2> 1676971 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1676991 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677021 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677138 DEBUG (ScheduledTrigger-4751-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677141 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677165 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677192 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677213 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677239 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677273 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677299 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677334 DEBUG (ScheduledTrigger-4760-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1677346 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[71E476A51D330AAC]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [testTrigger_collection]
[junit4] 2> DocCollection(testTrigger_collection//clusterstate.json/28)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 4
[junit4] 2> ## * testTrigger_collection 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - update 26
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1677347 DEBUG (TEST-IndexSizeTriggerTest.testTrigger-seed#[71E476A51D330AAC]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1677347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1677347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1677347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1677347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1677449 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[71E476A51D330AAC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testTrigger -Dtests.seed=71E476A51D330AAC -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=el-GR -Dtests.timezone=America/Anguilla -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 1.21s J1 | IndexSizeTriggerTest.testTrigger <<<
[junit4] > Throwable #1: java.lang.AssertionError: waitFor not elapsed but produced an event
[junit4] > at __randomizedtesting.SeedInfo.seed([71E476A51D330AAC:122F402784FC7981]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1677967 INFO (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[71E476A51D330AAC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSplitIntegration
[junit4] 2> 1678024 DEBUG (simCloudManagerPool-4759-thread-39) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1678025 DEBUG (simCloudManagerPool-4759-thread-40) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1678029 DEBUG (simCloudManagerPool-4759-thread-41) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1678029 DEBUG (simCloudManagerPool-4759-thread-41) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1678030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1678030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1678030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1678030 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1678032 DEBUG (simCloudManagerPool-4759-thread-42) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1678032 DEBUG (simCloudManagerPool-4759-thread-42) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1678032 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1678033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1678033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1678033 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1678034 DEBUG (simCloudManagerPool-4759-thread-43) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1678034 DEBUG (simCloudManagerPool-4759-thread-43) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1678034 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1678034 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1678034 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1678034 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1678142 DEBUG (ScheduledTrigger-4751-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1678641 DEBUG (simCloudManagerPool-4759-thread-95) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1678642 DEBUG (simCloudManagerPool-4759-thread-95) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1678642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1678642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1678642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1678642 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1678725 DEBUG (ScheduledTrigger-4760-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}
[junit4] 2> 1678730 DEBUG (ScheduledTrigger-4760-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger2]
[junit4] 2> 1678732 INFO (ScheduledTrigger-4760-thread-3) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210049583001060, stage=STARTED, actionName='null', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1678812 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}
[junit4] 2> 1678814 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210053689707360, stage=BEFORE_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1678814 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 1678829 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
[junit4] 2> 1678829 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
[junit4] 2> 1678830 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 27210054493 sessionWrapper.createTime 27210054412716610, this.sessionWrapper.createTime 27210054412716610
[junit4] 2> 1678830 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210054496071960, stage=AFTER_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":2,
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1678830 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210054519754610, stage=BEFORE_ACTION, actionName='execute_plan', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":2,
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1678831 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":2,
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@5f642a90, org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@5516c65f], AFTER_ACTION=[compute_plan], BEFORE_ACTION=[compute_plan, execute_plan]}
[junit4] 2> 1678831 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
[junit4] 2> 1679040 DEBUG (simCloudManagerPool-4759-thread-100) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_1: {"core_node10":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12,
[junit4] 2> "core":"testSplitIntegration_collection_shard1_1_replica1",
[junit4] 2> "shard":"shard1_1",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1679041 DEBUG (simCloudManagerPool-4759-thread-101) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_0: {"core_node7":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10001/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13,
[junit4] 2> "core":"testSplitIntegration_collection_shard1_0_replica0",
[junit4] 2> "shard":"shard1_0",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1679051 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
[junit4] 2> 1679142 DEBUG (ScheduledTrigger-4751-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1679427 DEBUG (simCloudManagerPool-4759-thread-107) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_1: {"core_node16":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12,
[junit4] 2> "core":"testSplitIntegration_collection_shard2_1_replica1",
[junit4] 2> "shard":"shard2_1",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1679457 DEBUG (simCloudManagerPool-4759-thread-110) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_0: {"core_node14":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13,
[junit4] 2> "core":"testSplitIntegration_collection_shard2_0_replica1",
[junit4] 2> "shard":"shard2_0",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 1679474 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210086707522910, stage=AFTER_ACTION, actionName='execute_plan', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":2,
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2/0] in completed tasks},success=}, {status={state=completed,msg=found [index_size_trigger2/60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1679475 INFO (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=27210086752287960, stage=SUCCEEDED, actionName='null', event={
[junit4] 2> "id":"60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2",
[junit4] 2> "source":"index_size_trigger2",
[junit4] 2> "eventTime":27210045141248460,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":2,
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":25,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":25,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":25,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "_enqueue_time_":27210049528850960,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}, context={}, config={
[junit4] 2> "trigger":"index_size_trigger2",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 1679479 INFO (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[71E476A51D330AAC]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [testSplitIntegration_collection]
[junit4] 2> DocCollection(testSplitIntegration_collection//clusterstate.json/87)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "stateTimestamp":"1526958175841502100",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":25,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":25},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":25,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":25}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"inactive"},
[junit4] 2> "shard1":{
[junit4] 2> "stateTimestamp":"1526958165578083950",
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":25,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":25},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":25,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":25}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"inactive"},
[junit4] 2> "shard2_1":{
[junit4] 2> "parent":"shard2",
[junit4] 2> "stateTimestamp":"1526958175842331850",
[junit4] 2> "range":"40000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node16":{
[junit4] 2> "leader":"true",
[junit4] 2> "core":"testSplitIntegration_collection_shard2_1_replica1",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12},
[junit4] 2> "core_node15":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_1_replica0",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10001/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12}}},
[junit4] 2> "shard1_1":{
[junit4] 2> "parent":"shard1",
[junit4] 2> "stateTimestamp":"1526958165614132950",
[junit4] 2> "range":"c0000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node10":{
[junit4] 2> "leader":"true",
[junit4] 2> "core":"testSplitIntegration_collection_shard1_1_replica1",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12},
[junit4] 2> "core_node9":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_1_replica0",
[junit4] 2> "SEARCHER.searcher.maxDoc":12,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10001/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":12}}},
[junit4] 2> "shard1_0":{
[junit4] 2> "parent":"shard1",
[junit4] 2> "stateTimestamp":"1526958165613807700",
[junit4] 2> "range":"80000000-bfffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node7":{
[junit4] 2> "leader":"true",
[junit4] 2> "core":"testSplitIntegration_collection_shard1_0_replica0",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10001/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13},
[junit4] 2> "core_node8":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_0_replica1",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13}}},
[junit4] 2> "shard2_0":{
[junit4] 2> "parent":"shard2",
[junit4] 2> "stateTimestamp":"1526958175842122000",
[junit4] 2> "range":"0-3fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node14":{
[junit4] 2> "leader":"true",
[junit4] 2> "core":"testSplitIntegration_collection_shard2_0_replica1",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13},
[junit4] 2> "core_node13":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_0_replica0",
[junit4] 2> "SEARCHER.searcher.maxDoc":13,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10001/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":13}}}}}
[junit4] 2> ## Max replicas per node: 6
[junit4] 2> ## Min replicas per node: 6
[junit4] 2> ## Total replicas: 12
[junit4] 2> ## * testSplitIntegration_collection 12
[junit4] 2> ## - active 12
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETESTATUS 2
[junit4] 2> ## - REQUESTSTATUS 2
[junit4] 2> ## - SPLITSHARD 2
[junit4] 2> ## - autoscaling 4
[junit4] 2> ## - update 52
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: index_size_trigger2
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 1679486 DEBUG (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[71E476A51D330AAC]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1679489 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1679497 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1679497 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1679497 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1679497 DEBUG (AutoscalingActionExecutor-4761-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 34570 ms for event id=60ab6452f769ccTbl5roldajdg6d50yrfyoifyn2
[junit4] 2> 1679724 INFO (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[71E476A51D330AAC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSplitIntegration
[junit4] 2> 1680007 INFO (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[71E476A51D330AAC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMergeIntegration
[junit4] 2> 1680149 DEBUG (ScheduledTrigger-4751-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1680183 DEBUG (simCloudManagerPool-4759-thread-119) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testMergeIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 1680184 DEBUG (simCloudManagerPool-4759-thread-120) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testMergeIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 1680316 DEBUG (simCloudManagerPool-4759-thread-132) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1680316 DEBUG (simCloudManagerPool-4759-thread-132) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1680317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1680317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1680317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1680317 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1680329 DEBUG (simCloudManagerPool-4759-thread-133) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1680329 DEBUG (simCloudManagerPool-4759-thread-133) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1680333 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1680333 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1680333 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1680333 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1680337 DEBUG (simCloudManagerPool-4759-thread-134) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1680337 DEBUG (simCloudManagerPool-4759-thread-134) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1680345 DEBU
[...truncated too long message...]
:: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null961335151
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null961335151
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null961335151
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null698654712
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null698654712
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null698654712
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 29ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 286 minutes 31 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)