You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/06/14 06:41:34 UTC
[JENKINS] Lucene-Solr-BadApples-7.x-Linux (64bit/jdk-10.0.1) -
Build # 50 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/50/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC
6 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:8BCFB730649B71B1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/133)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1528958556866825300", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528958556867176600", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528958556867112250", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/133)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528958556866825300",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528958556867176600",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528958556867112250",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:D18A38F2D2ABCB62]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testMixedBounds_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testMixedBounds_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1528958482236382850", "replicas":{ "core_node1":{ "core":"testMixedBounds_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}, "core_node2":{ "core":"testMixedBounds_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528958482251234700", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testMixedBounds_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}, "core_node9":{ "core":"testMixedBounds_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528958482251013800", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testMixedBounds_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}, "core_node8":{ "core":"testMixedBounds_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}}}}}
Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528958482236382850",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528958482251234700",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528958482251013800",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:E2873E1FB0EF09C6]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:559)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528958482236382850",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528958482251234700",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528958482251013800",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
... 39 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/134)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1528958648794876250", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528958648795225450", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528958648795165200", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/134)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528958648794876250",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528958648795225450",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528958648795165200",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:D18A38F2D2ABCB62]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/128)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1528958752484693450", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528958752485703100", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528958752485634350", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/128)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528958752484693450",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528958752485703100",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528958752485634350",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:D18A38F2D2ABCB62]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/132)={ "replicationFactor":"2", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10129_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10128_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1528976078786538600", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10129_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10128_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1528976078787305350", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10128_solr", "base_url":"http://127.0.0.1:10128/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10129_solr", "base_url":"http://127.0.0.1:10129/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1528976078787178950", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10129_solr", "base_url":"http://127.0.0.1:10129/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10128_solr", "base_url":"http://127.0.0.1:10128/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/132)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10129_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10128_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1528976078786538600",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10129_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10128_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1528976078787305350",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10128_solr",
"base_url":"http://127.0.0.1:10128/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10129_solr",
"base_url":"http://127.0.0.1:10129/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1528976078787178950",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10129_solr",
"base_url":"http://127.0.0.1:10129/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10128_solr",
"base_url":"http://127.0.0.1:10128/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([E80481B2FD54029C:D18A38F2D2ABCB62]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12884 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E80481B2FD54029C-001/init-core-data-001
[junit4] 2> 664309 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 664309 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 664310 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 664310 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E80481B2FD54029C-001/tempDir-001
[junit4] 2> 664310 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 664320 INFO (Thread-3626) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 664320 INFO (Thread-3626) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 664356 ERROR (Thread-3626) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 664420 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42259
[junit4] 2> 664444 INFO (zkConnectionManagerCallback-1844-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664452 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10.0.1+10
[junit4] 2> 664453 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10.0.1+10
[junit4] 2> 664471 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 664471 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 664471 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 664485 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62968a2b{/solr,null,AVAILABLE}
[junit4] 2> 664486 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2ca57f0a{SSL,[ssl, http/1.1]}{127.0.0.1:Û´Û³Û°Û¶Û³}
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.e.j.s.Server Started @Û¶Û¶Û´ÛµÛ°Û¸ms
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43063}
[junit4] 2> 664487 ERROR (jetty-launcher-1841-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solrâ„¢ version 7.5.0
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 664487 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-14T06:00:18.094544Z
[junit4] 2> 664504 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 664504 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 664504 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@651413b5{/solr,null,AVAILABLE}
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f59bab{SSL,[ssl, http/1.1]}{127.0.0.1:Û´Û±Û¹Û¸Û¹}
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.e.j.s.Server Started @Û¶Û¶Û´ÛµÛ²Û·ms
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41989}
[junit4] 2> 664505 ERROR (jetty-launcher-1841-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 664505 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solrâ„¢ version 7.5.0
[junit4] 2> 664506 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 664506 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 664506 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-14T06:00:18.113070Z
[junit4] 2> 664507 INFO (zkConnectionManagerCallback-1846-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664507 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 664533 INFO (zkConnectionManagerCallback-1848-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664533 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 664595 INFO (jetty-launcher-1841-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42259/solr
[junit4] 2> 664624 INFO (zkConnectionManagerCallback-1852-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664632 INFO (zkConnectionManagerCallback-1854-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664657 INFO (jetty-launcher-1841-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42259/solr
[junit4] 2> 664676 INFO (zkConnectionManagerCallback-1860-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664697 INFO (zkConnectionManagerCallback-1862-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664702 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 664702 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41989_solr
[junit4] 2> 664703 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.Overseer Overseer (id=72197652964704262-127.0.0.1:41989_solr-n_0000000000) starting
[junit4] 2> 664718 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 664719 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43063_solr
[junit4] 2> 664728 INFO (zkCallback-1861-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 664735 INFO (zkCallback-1853-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 664746 INFO (zkConnectionManagerCallback-1869-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664747 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 664747 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42259/solr ready
[junit4] 2> 664761 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 664762 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 664763 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 664763 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41989_solr
[junit4] 2> 664764 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 664765 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 664768 INFO (zkCallback-1853-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 664777 INFO (zkConnectionManagerCallback-1874-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664777 INFO (zkCallback-1861-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 664777 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 664778 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42259/solr ready
[junit4] 2> 664782 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:43063_solr]
[junit4] 2> 664796 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 664797 INFO (zkCallback-1868-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 664797 DEBUG (OverseerAutoScalingTriggerThread-72197652964704262-127.0.0.1:41989_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 664797 DEBUG (ScheduledTrigger-2693-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 664807 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 664827 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43063.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664835 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43063.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664836 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43063.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664836 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 664837 INFO (jetty-launcher-1841-thread-1) [n:127.0.0.1:43063_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E80481B2FD54029C-001/tempDir-001/node1/.
[junit4] 2> 664849 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41989.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664869 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41989.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664869 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41989.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3db2651b
[junit4] 2> 664870 INFO (jetty-launcher-1841-thread-2) [n:127.0.0.1:41989_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E80481B2FD54029C-001/tempDir-001/node2/.
[junit4] 2> 664897 INFO (zkConnectionManagerCallback-1878-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664908 INFO (zkConnectionManagerCallback-1883-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 664909 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 664909 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42259/solr ready
[junit4] 2> 664936 DEBUG (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10128_solr
[junit4] 2> 664937 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 664937 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 664937 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Curre
[...truncated too long message...]
[junit4] 2> 13172 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14689f33: rootName = solr_35319, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@43cd5137
[junit4] 2> 13172 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d1bc303: rootName = solr_43359, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2c96762b
[junit4] 2> 13177 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 13177 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e3ceb37: rootName = solr_35319, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@582889e2
[junit4] 2> 13182 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 13182 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@77c9b38f: rootName = solr_35319, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@d83dc3c
[junit4] 2> 13183 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 13183 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@50c3e81a: rootName = solr_43359, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@582889e2
[junit4] 2> 13183 INFO (jetty-closer-45-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35319_solr
[junit4] 2> 13185 INFO (jetty-closer-45-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 13185 INFO (zkCallback-64-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 13185 INFO (zkCallback-56-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 13185 INFO (zkCallback-71-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 13188 INFO (zkCallback-77-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 13188 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 13189 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7a6f2ae: rootName = solr_43359, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@d83dc3c
[junit4] 2> 13189 INFO (jetty-closer-45-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6ac344b1{/solr,null,UNAVAILABLE}
[junit4] 2> 13189 INFO (jetty-closer-45-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 13190 INFO (jetty-closer-45-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43359_solr
[junit4] 2> 13191 INFO (jetty-closer-45-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 13191 INFO (jetty-closer-45-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 13191 INFO (zkCallback-71-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 13191 INFO (zkCallback-64-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 13192 INFO (jetty-closer-45-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72197811028819974-127.0.0.1:43359_solr-n_0000000000) closing
[junit4] 2> 13192 INFO (OverseerStateUpdate-72197811028819974-127.0.0.1:43359_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43359_solr
[junit4] 2> 13192 WARN (OverseerAutoScalingTriggerThread-72197811028819974-127.0.0.1:43359_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 13195 DEBUG (jetty-closer-45-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 13196 INFO (jetty-closer-45-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@dd0e5dc{/solr,null,UNAVAILABLE}
[junit4] 2> 13196 INFO (jetty-closer-45-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 13197 ERROR (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 13197 INFO (SUITE-IndexSizeTriggerTest-seed#[E80481B2FD54029C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36045 36045
[junit4] 2> 13321 INFO (Thread-20) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36045 36045
[junit4] 2> 13321 WARN (Thread-20) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E80481B2FD54029C-002
[junit4] 2> Jun 14, 2018 6:40:33 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1680, maxMBSortInHeap=6.393579083888666, sim=RandomSimilarity(queryNorm=true): {}, locale=fa-IR, timezone=Australia/LHI
[junit4] 2> NOTE: Linux 4.13.0-41-generic amd64/Oracle Corporation 10.0.1 (64-bit)/cpus=8,threads=1,free=424403200,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [IndexSizeTriggerTest, IndexSizeTriggerTest]
[junit4] Completed [4/5 (3!)] on J1 in 3.81s, 4 tests, 1 error <<< FAILURES!
[...truncated 22 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 20 tests, 3 errors, 2 failures [seed: E80481B2FD54029C]
Total time: 17 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 3/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=35a7e95bce54d53820ce3e0b1e2966f609a1c1d2, workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #49
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS-EA] Lucene-Solr-BadApples-7.x-Linux (64bit/jdk-11-ea+14) -
Build # 53 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/53/
Java: 64bit/jdk-11-ea+14 -XX:-UseCompressedOops -XX:+UseSerialGC
9 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:43183/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:42907/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43183/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:42907/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:F38426010538E0B]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:288)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43183/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 45 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10003_solr, 127.0.0.1:10002_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/26)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testMixedBounds_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testMixedBounds_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1529485711559648900", "replicas":{ "core_node1":{ "core":"testMixedBounds_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}, "core_node2":{ "core":"testMixedBounds_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1529485711560061900", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testMixedBounds_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}, "core_node9":{ "core":"testMixedBounds_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1529485711559981200", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testMixedBounds_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}, "core_node8":{ "core":"testMixedBounds_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}}}}}
Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10003_solr, 127.0.0.1:10002_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/26)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529485711559648900",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529485711560061900",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529485711559981200",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:AF762E3FEA3B5081]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:561)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/26)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529485711559648900",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529485711560061900",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529485711559981200",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
... 39 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/66)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1529485827755713700", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1529485827756040850", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1529485827755980050", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10003_solr", "base_url":"http://127.0.0.1:10003/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10002_solr", "base_url":"http://127.0.0.1:10002/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/66)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529485827755713700",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10003_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529485827756040850",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529485827755980050",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10003_solr",
"base_url":"http://127.0.0.1:10003/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10002_solr",
"base_url":"http://127.0.0.1:10002/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:9C7B28D2887F9225]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:301)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/27)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testMixedBounds_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10005_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testMixedBounds_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10004_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1529485957954308300", "replicas":{ "core_node1":{ "core":"testMixedBounds_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10005_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}, "core_node2":{ "core":"testMixedBounds_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":495, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":257740, "node_name":"127.0.0.1:10004_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.4003908038139343E-4, "SEARCHER.searcher.numDocs":495}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1529485957954629200", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testMixedBounds_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10005_solr", "base_url":"http://127.0.0.1:10005/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}, "core_node9":{ "core":"testMixedBounds_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":247, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":133740, "node_name":"127.0.0.1:10004_solr", "base_url":"http://127.0.0.1:10004/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.245550811290741E-4, "SEARCHER.searcher.numDocs":247}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1529485957954567800", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testMixedBounds_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10004_solr", "base_url":"http://127.0.0.1:10004/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}, "core_node8":{ "core":"testMixedBounds_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":248, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":144480, "node_name":"127.0.0.1:10005_solr", "base_url":"http://127.0.0.1:10005/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.3455748558044434E-4, "SEARCHER.searcher.numDocs":248}}}}}
Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/27)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10005_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529485957954308300",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10005_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529485957954629200",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10005_solr",
"base_url":"http://127.0.0.1:10005/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529485957954567800",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10005_solr",
"base_url":"http://127.0.0.1:10005/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:AF762E3FEA3B5081]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:561)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/27)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testMixedBounds_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10005_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testMixedBounds_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529485957954308300",
"replicas":{
"core_node1":{
"core":"testMixedBounds_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10005_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495},
"core_node2":{
"core":"testMixedBounds_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":495,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":257740,
"node_name":"127.0.0.1:10004_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.4003908038139343E-4,
"SEARCHER.searcher.numDocs":495}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529485957954629200",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10005_solr",
"base_url":"http://127.0.0.1:10005/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247},
"core_node9":{
"core":"testMixedBounds_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":247,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":133740,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.245550811290741E-4,
"SEARCHER.searcher.numDocs":247}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529485957954567800",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testMixedBounds_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10004_solr",
"base_url":"http://127.0.0.1:10004/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248},
"core_node8":{
"core":"testMixedBounds_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":248,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":144480,
"node_name":"127.0.0.1:10005_solr",
"base_url":"http://127.0.0.1:10005/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.3455748558044434E-4,
"SEARCHER.searcher.numDocs":248}}}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
... 39 more
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:C63EA7103E4F28F6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/69)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1529486353691402950", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1529486353691757000", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10007_solr", "base_url":"http://127.0.0.1:10007/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10006_solr", "base_url":"http://127.0.0.1:10006/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1529486353691698500", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10006_solr", "base_url":"http://127.0.0.1:10006/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10007_solr", "base_url":"http://127.0.0.1:10007/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/69)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10006_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529486353691402950",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10006_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529486353691757000",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10007_solr",
"base_url":"http://127.0.0.1:10007/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10006_solr",
"base_url":"http://127.0.0.1:10006/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529486353691698500",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10006_solr",
"base_url":"http://127.0.0.1:10006/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10007_solr",
"base_url":"http://127.0.0.1:10007/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:9C7B28D2887F9225]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:301)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/87)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}, "core_node4":{ "core":"testSplitIntegration_collection_shard2_replica_n4", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.4659017324447632E-5, "SEARCHER.searcher.numDocs":11}}, "range":"0-7fffffff", "state":"active"}, "shard1":{ "stateTimestamp":"1529481030895064250", "replicas":{ "core_node1":{ "core":"testSplitIntegration_collection_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}, "core_node2":{ "core":"testSplitIntegration_collection_shard1_replica_n2", "SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":17240, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.605600118637085E-5, "SEARCHER.searcher.numDocs":14}}, "range":"80000000-ffffffff", "state":"inactive"}, "shard1_1":{ "parent":"shard1", "stateTimestamp":"1529481030895541450", "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node10":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_1_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}, "core_node9":{ "core":"testSplitIntegration_collection_shard1_1_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":13740, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":1.2796372175216675E-5, "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{ "parent":"shard1", "stateTimestamp":"1529481030895460100", "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node7":{ "leader":"true", "core":"testSplitIntegration_collection_shard1_0_replica0", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10001_solr", "base_url":"http://127.0.0.1:10001/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}, "core_node8":{ "core":"testSplitIntegration_collection_shard1_0_replica1", "SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":23980, "node_name":"127.0.0.1:10000_solr", "base_url":"http://127.0.0.1:10000/solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":2.2333115339279175E-5, "SEARCHER.searcher.numDocs":7}}}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/87)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11},
"core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":15740,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.4659017324447632E-5,
"SEARCHER.searcher.numDocs":11}},
"range":"0-7fffffff",
"state":"active"},
"shard1":{
"stateTimestamp":"1529481030895064250",
"replicas":{
"core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14},
"core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":17240,
"node_name":"127.0.0.1:10000_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.605600118637085E-5,
"SEARCHER.searcher.numDocs":14}},
"range":"80000000-ffffffff",
"state":"inactive"},
"shard1_1":{
"parent":"shard1",
"stateTimestamp":"1529481030895541450",
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":13740,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":1.2796372175216675E-5,
"SEARCHER.searcher.numDocs":7}}},
"shard1_0":{
"parent":"shard1",
"stateTimestamp":"1529481030895460100",
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node7":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":23980,
"node_name":"127.0.0.1:10000_solr",
"base_url":"http://127.0.0.1:10000/solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":2.2333115339279175E-5,
"SEARCHER.searcher.numDocs":7}}}}}
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:9C7B28D2887F9225]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:301)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
number of ops expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([A5F59192A7805BDB:C63EA7103E4F28F6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:832)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/74)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard2":{ "replicas":{ "core_node3":{ "core":"testSplitIntegration_collection_shard2_replica_n3", "leader":"true", "SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":15740, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":
[...truncated too long message...]
cManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 116766 INFO (zkCallback-290-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 116766 INFO (zkCallback-277-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 116767 INFO (jetty-closer-258-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72232058244956166-127.0.0.1:36935_solr-n_0000000000) closing
[junit4] 2> 116767 INFO (OverseerStateUpdate-72232058244956166-127.0.0.1:36935_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36935_solr
[junit4] 2> 116767 WARN (OverseerAutoScalingTriggerThread-72232058244956166-127.0.0.1:36935_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 116767 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 116768 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 116768 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 116768 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 116768 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 116768 DEBUG (jetty-closer-258-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 116769 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40917_solr
[junit4] 2> 116769 INFO (jetty-closer-258-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7c2729fd{/solr,null,UNAVAILABLE}
[junit4] 2> 116769 INFO (jetty-closer-258-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 116771 INFO (jetty-closer-258-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 116771 INFO (jetty-closer-258-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f320505: rootName = solr_40917, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@42a5af82
[junit4] 2> 116772 INFO (jetty-closer-258-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40917_solr
[junit4] 2> 116773 INFO (jetty-closer-258-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 116773 INFO (jetty-closer-258-thread-2) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 116773 INFO (zkCallback-290-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 116773 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 116774 INFO (jetty-closer-258-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@32fbb51b{/solr,null,UNAVAILABLE}
[junit4] 2> 116774 INFO (jetty-closer-258-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 116774 ERROR (SUITE-IndexSizeTriggerTest-seed#[A5F59192A7805BDB]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 116775 INFO (SUITE-IndexSizeTriggerTest-seed#[A5F59192A7805BDB]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42753 42753
[junit4] 2> 117192 INFO (Thread-1620) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42753 42753
[junit4] 2> 117192 WARN (Thread-1620) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_A5F59192A7805BDB-004
[junit4] 2> Jun 20, 2018 7:50:04 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=2004, maxMBSortInHeap=6.7462155014282565, sim=RandomSimilarity(queryNorm=false): {}, locale=en-IN, timezone=UTC
[junit4] 2> NOTE: Linux 4.13.0-41-generic amd64/Oracle Corporation 11-ea (64-bit)/cpus=8,threads=1,free=177547288,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [IndexSizeTriggerTest, MoveReplicaHDFSTest, IndexSizeTriggerTest, IndexSizeTriggerTest, IndexSizeTriggerTest]
[junit4] Completed [7/10 (4!)] on J2 in 3.49s, 4 tests, 1 error <<< FAILURES!
[junit4]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] OK 0.78s J2 | IndexSizeTriggerTest.testMixedBounds
[junit4] OK 0.67s J2 | IndexSizeTriggerTest.testSplitIntegration
[junit4] OK 0.30s J2 | IndexSizeTriggerTest.testTrigger
[junit4] OK 0.34s J2 | IndexSizeTriggerTest.testMergeIntegration
[junit4] Completed [8/10 (4!)] on J2 in 5.14s, 4 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] OK 18.3s J1 | MoveReplicaHDFSTest.testNormalMove
[junit4] OK 20.6s J1 | MoveReplicaHDFSTest.test
[junit4] OK 13.2s J1 | MoveReplicaHDFSTest.testFailedMove
[junit4] OK 12.7s J1 | MoveReplicaHDFSTest.testNormalFailedMove
[junit4] Completed [9/10 (4!)] on J1 in 81.19s, 4 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] OK 19.2s J0 | MoveReplicaHDFSTest.testNormalMove
[junit4] OK 20.6s J0 | MoveReplicaHDFSTest.test
[junit4] OK 15.7s J0 | MoveReplicaHDFSTest.testFailedMove
[junit4] OK 12.5s J0 | MoveReplicaHDFSTest.testNormalFailedMove
[junit4] Completed [10/10 (4!)] on J0 in 80.96s, 4 tests
[junit4]
[junit4]
[junit4] Tests with failures [seed: A5F59192A7805BDB]:
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
[junit4] - org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
[junit4]
[junit4]
[junit4] JVM J0: 0.80 .. 183.43 = 182.63s
[junit4] JVM J1: 0.79 .. 177.41 = 176.62s
[junit4] JVM J2: 0.82 .. 123.32 = 122.49s
[junit4] Execution time total: 3 minutes 3 seconds
[junit4] Tests summary: 10 suites, 40 tests, 3 errors, 4 failures
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 40 tests, 3 errors, 4 failures [seed: A5F59192A7805BDB]
Total time: 3 minutes 5 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 0/5 failed: org.apache.solr.cloud.MoveReplicaHDFSTest
[repro] 4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=9efe619afa636d18362f38ba651bf8bc7244c6a9, workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #52
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-BadApples-7.x-Linux (32bit/jdk1.8.0_172) -
Build # 52 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/52/
Java: 32bit/jdk1.8.0_172 -client -XX:+UseSerialGC
7 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [ZkCollectionTerms] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkCollectionTerms at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkCollectionTerms.<init>(ZkCollectionTerms.java:39) at org.apache.solr.cloud.ZkController.getCollectionTerms(ZkController.java:1505) at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1524) at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1518) at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1516) at org.apache.solr.cloud.ZkController$UnloadCoreOnDeletedWatcher.onStateChanged(ZkController.java:2746) at org.apache.solr.common.cloud.ZkStateReader$Notification.run(ZkStateReader.java:1642) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [ZkCollectionTerms]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkCollectionTerms
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.cloud.ZkCollectionTerms.<init>(ZkCollectionTerms.java:39)
at org.apache.solr.cloud.ZkController.getCollectionTerms(ZkController.java:1505)
at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1524)
at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1518)
at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1516)
at org.apache.solr.cloud.ZkController$UnloadCoreOnDeletedWatcher.onStateChanged(ZkController.java:2746)
at org.apache.solr.common.cloud.ZkStateReader$Notification.run(ZkStateReader.java:1642)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:304)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
did not finish processing all events in time: started=3, finished=2
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=3, finished=2
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:F339A5FBAA7BABDF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:253)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
did not finish processing all events in time: started=4, finished=3
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=4, finished=3
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:F339A5FBAA7BABDF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:253)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:E7D67610F64D4877]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:E7D67610F64D4877]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:E7D67610F64D4877]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([58C3B8EE75A72DF1:E7D67610F64D4877]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12516 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/init-core-data-001
[junit4] 2> 65500 WARN (SUITE-HttpPartitionTest-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 65501 INFO (SUITE-HttpPartitionTest-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 65502 INFO (SUITE-HttpPartitionTest-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 65502 INFO (SUITE-HttpPartitionTest-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 65502 INFO (SUITE-HttpPartitionTest-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jr_i/k
[junit4] 2> 65504 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 65504 INFO (Thread-134) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 65504 INFO (Thread-134) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 65506 ERROR (Thread-134) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 65604 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ZkTestServer start zk server on port:33613
[junit4] 2> 65607 INFO (zkConnectionManagerCallback-142-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 65618 INFO (zkConnectionManagerCallback-144-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 65624 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 65626 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 65633 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 65635 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 65636 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 65637 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 65639 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 65640 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 65642 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 65643 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 65644 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 65645 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 65726 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 65727 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 65727 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 65727 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 65727 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f39877{/jr_i/k,null,AVAILABLE}
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3a1623{HTTP/1.1,[http/1.1]}{127.0.0.1:46005}
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server Started @65766ms
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/tempDir-001/control/data, hostContext=/jr_i/k, hostPort=44301, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/control-001/cores}
[junit4] 2> 65729 ERROR (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solrâ„¢ version 7.5.0
[junit4] 2> 65729 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 65730 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 65730 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-18T05:38:04.373Z
[junit4] 2> 65731 INFO (zkConnectionManagerCallback-146-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 65733 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 65733 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/control-001/solr.xml
[junit4] 2> 65740 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 65740 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 65744 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 67014 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33613/solr
[junit4] 2> 67017 INFO (zkConnectionManagerCallback-150-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 67021 INFO (zkConnectionManagerCallback-152-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 67088 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 67088 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44301_jr_i%2Fk
[junit4] 2> 67089 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.Overseer Overseer (id=72220214795173892-127.0.0.1:44301_jr_i%2Fk-n_0000000000) starting
[junit4] 2> 67094 INFO (zkConnectionManagerCallback-159-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 67096 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33613/solr ready
[junit4] 2> 67098 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44301_jr_i%2Fk
[junit4] 2> 67099 INFO (zkCallback-151-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 67099 INFO (zkCallback-158-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 67117 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 67141 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 67151 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 67152 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 67153 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/control-001/cores
[junit4] 2> 67174 INFO (zkConnectionManagerCallback-165-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 67175 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 67176 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33613/solr ready
[junit4] 2> 67177 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42126,localport=44301], receiveBufferSize:531000
[junit4] 2> 67181 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46005,localport=42432], receiveBufferSize=530904
[junit4] 2> 67182 INFO (qtp32117015-617) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44301_jr_i%252Fk&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 67184 INFO (OverseerThreadFactory-319-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 67291 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42130,localport=44301], receiveBufferSize:531000
[junit4] 2> 67292 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46005,localport=42436], receiveBufferSize=530904
[junit4] 2> 67294 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 67294 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 68328 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 68373 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 68508 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 68532 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 68532 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 68532 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 68532 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 68536 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@cc3a98
[junit4] 2> 68538 WARN (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 68596 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 68596 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 68598 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 68598 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 68599 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=78.1484375, floorSegmentMB=0.9306640625, forceMergeDeletesPctAllowed=7.338954919323935, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, reclaimDeletesWeight=0.17701508674919264
[junit4] 2> 68600 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@805f8c[control_collection_shard1_replica_n1] main]
[junit4] 2> 68601 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 68602 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 68602 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 68604 INFO (searcherExecutor-324-thread-1-processing-n:127.0.0.1:44301_jr_i%2Fk x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@805f8c[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 68604 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1603587578000310272
[junit4] 2> 68610 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44301/jr_i/k/control_collection_shard1_replica_n1/
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:44301/jr_i/k/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 68613 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 68615 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44301/jr_i/k/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 68768 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 68791 INFO (qtp32117015-612) [n:127.0.0.1:44301_jr_i%2Fk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1497
[junit4] 2> 68794 INFO (qtp32117015-617) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 68872 INFO (zkCallback-151-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 69186 INFO (OverseerCollectionConfigSetProcessor-72220214795173892-127.0.0.1:44301_jr_i%2Fk-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 69795 INFO (qtp32117015-617) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44301_jr_i%252Fk&wt=javabin&version=2} status=0 QTime=2613
[junit4] 2> 69801 INFO (zkConnectionManagerCallback-170-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 69803 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 69804 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33613/solr ready
[junit4] 2> 69804 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 69804 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42146,localport=44301], receiveBufferSize:531000
[junit4] 2> 69807 INFO (SocketProxy-Acceptor-44301) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46005,localport=42452], receiveBufferSize=530904
[junit4] 2> 69808 INFO (qtp32117015-613) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 69811 INFO (OverseerThreadFactory-319-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 69812 WARN (OverseerThreadFactory-319-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 70017 INFO (qtp32117015-613) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 70018 INFO (qtp32117015-613) [n:127.0.0.1:44301_jr_i%2Fk ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=210
[junit4] 2> 70101 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001 of type TLOG
[junit4] 2> 70101 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 70102 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 70102 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 70102 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 70103 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f8d51c{/jr_i/k,null,AVAILABLE}
[junit4] 2> 70103 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@f03172{HTTP/1.1,[http/1.1]}{127.0.0.1:40193}
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server Started @70141ms
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/jr_i/k, hostPort=33293, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001/cores}
[junit4] 2> 70104 ERROR (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solrâ„¢ version 7.5.0
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 70104 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-18T05:38:08.747Z
[junit4] 2> 70106 INFO (zkConnectionManagerCallback-172-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 70107 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 70107 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001/solr.xml
[junit4] 2> 70114 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 70114 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 70115 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 71724 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33613/solr
[junit4] 2> 71726 INFO (zkConnectionManagerCallback-176-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 71730 INFO (zkConnectionManagerCallback-178-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 71735 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 71736 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 71737 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 71738 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33293_jr_i%2Fk
[junit4] 2> 71738 INFO (zkCallback-158-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71738 INFO (zkCallback-151-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71739 INFO (zkCallback-169-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71739 INFO (zkCallback-177-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71756 INFO (zkConnectionManagerCallback-185-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 71757 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 71759 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33613/solr ready
[junit4] 2> 71759 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 71781 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 71803 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 71803 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 71805 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001/cores
[junit4] 2> 71813 INFO (OverseerCollectionConfigSetProcessor-72220214795173892-127.0.0.1:44301_jr_i%2Fk-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 71830 INFO (SocketProxy-Acceptor-33293) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=47434,localport=33293], receiveBufferSize:531000
[junit4] 2> 71831 INFO (SocketProxy-Acceptor-33293) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40193,localport=57746], receiveBufferSize=530904
[junit4] 2> 71832 INFO (qtp20324729-681) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33293_jr_i%252Fk&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 71836 INFO (OverseerThreadFactory-319-thread-3) [ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:33293_jr_i%2Fk for creating new replica
[junit4] 2> 71839 INFO (SocketProxy-Acceptor-33293) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=47438,localport=33293], receiveBufferSize:531000
[junit4] 2> 71839 INFO (SocketProxy-Acceptor-33293) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40193,localport=57750], receiveBufferSize=530904
[junit4] 2> 71841 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk x:collection1_shard2_replica_t41] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 72884 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 72908 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 73021 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 73039 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 73040 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 73040 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 73040 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
[junit4] 2> 73043 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@b46e7c
[junit4] 2> 73044 WARN (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 73098 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 73098 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 73100 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 73100 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 73101 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=78.1484375, floorSegmentMB=0.9306640625, forceMergeDeletesPctAllowed=7.338954919323935, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, reclaimDeletesWeight=0.17701508674919264
[junit4] 2> 73102 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@19d4fee[collection1_shard2_replica_t41] main]
[junit4] 2> 73103 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 73103 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 73104 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 73105 INFO (searcherExecutor-338-thread-1-processing-n:127.0.0.1:33293_jr_i%2Fk x:collection1_shard2_replica_t41 c:collection1 s:shard2) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@19d4fee[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 73105 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1603587582719950848
[junit4] 2> 73110 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node42=0}, version=0}
[junit4] 2> 73112 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 73112 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 73112 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33293/jr_i/k/collection1_shard2_replica_t41/
[junit4] 2> 73112 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 73113 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:33293/jr_i/k/collection1_shard2_replica_t41/ has no replicas
[junit4] 2> 73113 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 73113 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
[junit4] 2> 73115 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33293/jr_i/k/collection1_shard2_replica_t41/ shard2
[junit4] 2> 73217 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 73219 INFO (qtp20324729-676) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1378
[junit4] 2> 73222 INFO (qtp20324729-681) [n:127.0.0.1:33293_jr_i%2Fk c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:33293_jr_i%252Fk&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1389
[junit4] 2> 73311 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001 of type TLOG
[junit4] 2> 73311 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 73312 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 73312 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 73312 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 73313 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20a93e{/jr_i/k,null,AVAILABLE}
[junit4] 2> 73314 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1508a70{HTTP/1.1,[http/1.1]}{127.0.0.1:40207}
[junit4] 2> 73314 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.e.j.s.Server Started @73351ms
[junit4] 2> 73314 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/jr_i/k, hostPort=34749, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001/cores}
[junit4] 2> 73314 ERROR (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 73314 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 73314 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solrâ„¢ version 7.5.0
[junit4] 2> 73315 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 73315 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 73315 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-18T05:38:11.958Z
[junit4] 2> 73316 INFO (zkConnectionManagerCallback-188-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 73318 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 73318 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001/solr.xml
[junit4] 2> 73319 INFO (zkCallback-177-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 73322 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 73322 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 73323 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 73836 INFO (OverseerCollectionConfigSetProcessor-72220214795173892-127.0.0.1:44301_jr_i%2Fk-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 75047 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33613/solr
[junit4] 2> 75049 INFO (zkConnectionManagerCallback-192-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 75056 INFO (zkConnectionManagerCallback-194-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 75062 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 75064 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 75066 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 75066 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34749_jr_i%2Fk
[junit4] 2> 75067 INFO (zkCallback-169-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75067 INFO (zkCallback-158-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75067 INFO (zkCallback-177-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75067 INFO (zkCallback-151-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75069 INFO (zkCallback-184-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75069 INFO (zkCallback-193-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 75097 INFO (zkConnectionManagerCallback-201-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 75098 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 75099 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33613/solr ready
[junit4] 2> 75100 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 75121 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 75136 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 75136 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 75138 INFO (TEST-HttpPartitionTest.test-seed#[58C3B8EE75A72DF1]) [n:127.0.0.1:34749_jr_i%2Fk ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001/cores
[junit4] 2> 75175 INFO (qtp20324729-675) [n:127.0.0.1:33293_jr_i%2Fk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:34749_jr_i%252Fk&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 75179 INFO (OverseerThreadFactory-319-thread-4) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34749_jr_i%2Fk for creating new replica
[junit4] 2> 75181 INFO (SocketProxy-Acceptor-34749) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=52258,localport=34749], receiveBufferSize:531000
[junit4] 2> 75182 INFO (SocketProxy-Acceptor-34749) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40207,localport=46150], receiveBufferSize=530904
[junit4] 2> 75184 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk x:collection1_shard1_replica_t43] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 75288 INFO (zkCallback-177-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 76241 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 76260 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
[junit4] 2> 76421 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 76446 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
[junit4] 2> 76447 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ccf953
[junit4] 2> 76447 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 76447 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_58C3B8EE75A72DF1-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
[junit4] 2> 76450 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@ef686e
[junit4] 2> 76451 WARN (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 76511 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 76511 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 76513 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 76513 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 76515 INFO (qtp18928032-726) [n:127.0.0.1:34749_jr_i%2Fk c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=78.1484375, floorSegmentMB=0.9306640
[...truncated too long message...]
lr&inPlaceMove=true&replica=core_node124
[junit4] 2> 543689 INFO (AutoscalingActionExecutor-259-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2508894149 sessionWrapper.createTime 2508845123171295, this.sessionWrapper.createTime 2508845123171295
[junit4] 2> 543689 WARN (AutoscalingActionExecutor-259-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:790) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
[junit4] 2> ... 9 more
[junit4] 2> 543690 WARN (AutoscalingActionExecutor-259-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: compute for trigger event: {
[junit4] 2> "id":"8e9ba6e00120fT7xk3ke4qhab2gnoaz7b7n57mw",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":2508786732372495,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2508786732372495,
[junit4] 2> 2508786736050295,
[junit4] 2> 2508786734790295,
[junit4] 2> 2508786733066795,
[junit4] 2> 2508786733943445,
[junit4] 2> 2508786735633395,
[junit4] 2> 2508786733509945,
[junit4] 2> 2508786736471995,
[junit4] 2> 2508786735212945,
[junit4] 2> 2508786734365395,
[junit4] 2> 2508786736894195],
[junit4] 2> "_enqueue_time_":2508844916396495,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10282_solr",
[junit4] 2> "127.0.0.1:10286_solr",
[junit4] 2> "127.0.0.1:10278_solr",
[junit4] 2> "127.0.0.1:10279_solr",
[junit4] 2> "127.0.0.1:10284_solr",
[junit4] 2> "127.0.0.1:10287_solr",
[junit4] 2> "127.0.0.1:10281_solr",
[junit4] 2> "127.0.0.1:10280_solr",
[junit4] 2> "127.0.0.1:10288_solr",
[junit4] 2> "127.0.0.1:10285_solr",
[junit4] 2> "127.0.0.1:10283_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:312) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
[junit4] 2> "id":"8e9ba6e00120fT7xk3ke4qhab2gnoaz7b7n57mw",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":2508786732372495,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2508786732372495,
[junit4] 2> 2508786736050295,
[junit4] 2> 2508786734790295,
[junit4] 2> 2508786733066795,
[junit4] 2> 2508786733943445,
[junit4] 2> 2508786735633395,
[junit4] 2> 2508786733509945,
[junit4] 2> 2508786736471995,
[junit4] 2> 2508786735212945,
[junit4] 2> 2508786734365395,
[junit4] 2> 2508786736894195],
[junit4] 2> "_enqueue_time_":2508844916396495,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10282_solr",
[junit4] 2> "127.0.0.1:10286_solr",
[junit4] 2> "127.0.0.1:10278_solr",
[junit4] 2> "127.0.0.1:10279_solr",
[junit4] 2> "127.0.0.1:10284_solr",
[junit4] 2> "127.0.0.1:10287_solr",
[junit4] 2> "127.0.0.1:10281_solr",
[junit4] 2> "127.0.0.1:10280_solr",
[junit4] 2> "127.0.0.1:10288_solr",
[junit4] 2> "127.0.0.1:10285_solr",
[junit4] 2> "127.0.0.1:10283_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:156) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:309) ~[java/:?]
[junit4] 2> ... 6 more
[junit4] 2> Caused by: java.lang.InterruptedException: stopping - thread was interrupted
[junit4] 2> at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:108) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:309) ~[java/:?]
[junit4] 2> ... 6 more
[junit4] 2> 543690 DEBUG (AutoscalingActionExecutor-259-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 49178 ms for event id=8e9ba6e00120fT7xk3ke4qhab2gnoaz7b7n57mw
[junit4] 2> 543690 DEBUG (SUITE-TestLargeCluster-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 543690 DEBUG (SUITE-TestLargeCluster-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 543690 DEBUG (SUITE-TestLargeCluster-seed#[58C3B8EE75A72DF1]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestLargeCluster_58C3B8EE75A72DF1-002
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1597, maxMBSortInHeap=6.380549246985257, sim=RandomSimilarity(queryNorm=false): {}, locale=lv-LV, timezone=Asia/Gaza
[junit4] 2> NOTE: Linux 4.13.0-41-generic i386/Oracle Corporation 1.8.0_172 (32-bit)/cpus=8,threads=1,free=115361688,total=284688384
[junit4] 2> NOTE: All tests run in this JVM: [HttpPartitionTest, TestLargeCluster, TestLargeCluster]
[junit4] Completed [10/10 (5!)] on J0 in 229.27s, 4 tests, 1 failure <<< FAILURES!
[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 25 tests, 5 failures [seed: 58C3B8EE75A72DF1]
Total time: 9 minutes 6 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 0/5 failed: org.apache.solr.cloud.HttpPartitionTest
[repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=e5407c5a9710247e5f728aae36224a245a51f0b1, workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #51
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-BadApples-7.x-Linux (64bit/jdk-10.0.1) -
Build # 51 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/51/
Java: 64bit/jdk-10.0.1 -XX:-UseCompressedOops -XX:+UseG1GC
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([1961A9D5677685CF:A674672BE49CE049]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
did not finish processing all events in time: started=5, finished=4
Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
at __randomizedtesting.SeedInfo.seed([1961A9D5677685CF:A674672BE49CE049]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:522)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:375)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 13424 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_1961A9D5677685CF-001/init-core-data-001
[junit4] 2> 1025244 INFO (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1025245 INFO (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 1025245 INFO (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1025255 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1025255 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 1025255 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1025255 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1025255 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1025256 INFO (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1025257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10017_solr, 127.0.0.1:10002_solr, 127.0.0.1:10012_solr, 127.0.0.1:10007_solr, 127.0.0.1:10015_solr, 127.0.0.1:10006_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10005_solr, 127.0.0.1:10009_solr, 127.0.0.1:10013_solr, 127.0.0.1:10000_solr, 127.0.0.1:10008_solr, 127.0.0.1:10014_solr, 127.0.0.1:10003_solr, 127.0.0.1:10011_solr]
[junit4] 2> 1025257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1025257 DEBUG (ScheduledTrigger-6289-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 28
[junit4] 2> 1025262 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
[junit4] 2> 1025262 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1025263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1025263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1025263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1025263 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1025263 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1025264 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1025264 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1025264 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1025264 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1025264 DEBUG (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1025263 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2050) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2084) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:844) [?:?]
[junit4] 2> 1025264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1025264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1025264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1025264 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1025265 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1025265 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025265 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025265 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1025285 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025305 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025326 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025346 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025366 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1025371 DEBUG (simCloudManagerPool-6293-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1025371 DEBUG (simCloudManagerPool-6293-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1025371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1025371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1025375 DEBUG (simCloudManagerPool-6293-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1025375 DEBUG (simCloudManagerPool-6293-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1025375 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1025375 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1025375 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025375 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1025403 DEBUG (simCloudManagerPool-6293-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: {"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system"}}
[junit4] 2> 1025406 DEBUG (simCloudManagerPool-6293-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1025406 DEBUG (simCloudManagerPool-6293-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1025407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1025407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1025407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1025407 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025409 DEBUG (simCloudManagerPool-6293-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1025409 DEBUG (simCloudManagerPool-6293-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1025409 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1025409 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1025409 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1025427 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025448 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025468 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025488 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025508 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025529 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025549 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025569 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025589 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025610 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025630 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025650 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025670 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025691 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025711 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025731 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025751 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025772 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025792 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025812 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025832 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025853 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025873 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025893 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025913 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025934 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025954 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025974 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1025994 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026015 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026035 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026055 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026075 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026095 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026116 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026136 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026156 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026176 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026196 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026217 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026237 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026257 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026277 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
[junit4] 2> 1026280 DEBUG (simCloudManagerPool-6293-thread-85) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node54":{
[junit4] 2> "core":"testBasic_shard4_replica_t54",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10055_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026280 DEBUG (simCloudManagerPool-6293-thread-84) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node39":{
[junit4] 2> "core":"testBasic_shard3_replica_t39",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10049_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026280 DEBUG (simCloudManagerPool-6293-thread-83) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard2: {"core_node18":{
[junit4] 2> "core":"testBasic_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10052_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026280 DEBUG (simCloudManagerPool-6293-thread-86) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard5: {"core_node65":{
[junit4] 2> "core":"testBasic_shard5_replica_n65",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10087_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026280 DEBUG (simCloudManagerPool-6293-thread-87) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard1: {"core_node1":{
[junit4] 2> "core":"testBasic_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10054_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026291 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 618ms
[junit4] 2> 1026297 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 97
[junit4] 2> 1026298 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10082_solr
[junit4] 2> 1026298 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10077_solr
[junit4] 2> 1026298 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10026_solr
[junit4] 2> 1026318 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 94
[junit4] 2> 1026318 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10016_solr
[junit4] 2> 1026318 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10083_solr
[junit4] 2> 1026318 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10058_solr
[junit4] 2> 1026327 INFO (simCloudManagerPool-6293-thread-119) [ ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 2581ms
[junit4] 2> 1026338 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
[junit4] 2> 1026338 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10049_solr
[junit4] 2> 1026338 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10073_solr
[junit4] 2> 1026358 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
[junit4] 2> 1026379 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
[junit4] 2> 1026399 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
[junit4] 2> 1026399 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10082_solr, 127.0.0.1:10077_solr, 127.0.0.1:10026_solr]
[junit4] 2> 1026399 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"82ed6c9f3f457T9pw511u6t00f16ou7eh7prv38",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303299859772503,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303299859772503,
[junit4] 2> 2303299860425653,
[junit4] 2> 2303299860874453],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10082_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10026_solr"]}}
[junit4] 2> 1026399 DEBUG (ScheduledTrigger-6294-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
[junit4] 2> 1026402 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"82ed6c9f3f457T9pw511u6t00f16ou7eh7prv38",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303299859772503,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303299859772503,
[junit4] 2> 2303299860425653,
[junit4] 2> 2303299860874453],
[junit4] 2> "_enqueue_time_":2303304955681953,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10082_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10026_solr"]}}
[junit4] 2> 1026403 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"82ed6c9f3f457T9pw511u6t00f16ou7eh7prv38",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303299859772503,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303299859772503,
[junit4] 2> 2303299860425653,
[junit4] 2> 2303299860874453],
[junit4] 2> "_enqueue_time_":2303304955681953,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10082_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10026_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
[junit4] 2> 1026450 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node74
[junit4] 2> 1026476 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node29
[junit4] 2> 1026488 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node32
[junit4] 2> 1026488 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2303309395 sessionWrapper.createTime 2303305196939953, this.sessionWrapper.createTime 2303305196939953
[junit4] 2> 1026488 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"82ed6c9f3f457T9pw511u6t00f16ou7eh7prv38",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303299859772503,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303299859772503,
[junit4] 2> 2303299860425653,
[junit4] 2> 2303299860874453],
[junit4] 2> "_enqueue_time_":2303304955681953,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10082_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10026_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2d026946, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@770b3d32, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2cce3b32], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
[junit4] 2> 1026489 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node74
[junit4] 2> 1026589 DEBUG (simCloudManagerPool-6293-thread-124) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node77":{
[junit4] 2> "core":"testBasic_shard5_replica_p76",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10072_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 1026701 DEBUG (simCloudManagerPool-6293-thread-126) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node31":{
[junit4] 2> "core":"testBasic_shard3_replica_n31",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10074_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 1026702 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node29
[junit4] 2> 1026803 DEBUG (simCloudManagerPool-6293-thread-132) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node79":{
[junit4] 2> "core":"testBasic_shard2_replica_p78",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10056_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 1026915 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node32
[junit4] 2> 1027016 DEBUG (simCloudManagerPool-6293-thread-140) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node81":{
[junit4] 2> "core":"testBasic_shard3_replica_n80",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10059_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1027129 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 1027129 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 36348 ms for event id=82ed6c9f3f457T9pw511u6t00f16ou7eh7prv38
[junit4] 2> 1027230 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
[junit4] 2> 1027230 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10073_solr, 127.0.0.1:10016_solr, 127.0.0.1:10049_solr, 127.0.0.1:10058_solr, 127.0.0.1:10083_solr]
[junit4] 2> 1027230 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"82ed7436034b3T9pw511u6t00f16ou7eh7prv3b",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303301896910003,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303301896910003,
[junit4] 2> 2303300879474203,
[junit4] 2> 2303301896392503,
[junit4] 2> 2303300880739803,
[junit4] 2> 2303300880180203],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10073_solr",
[junit4] 2> "127.0.0.1:10016_solr",
[junit4] 2> "127.0.0.1:10049_solr",
[junit4] 2> "127.0.0.1:10058_solr",
[junit4] 2> "127.0.0.1:10083_solr"]}}
[junit4] 2> 1027230 DEBUG (ScheduledTrigger-6294-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
[junit4] 2> 1027232 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"82ed7436034b3T9pw511u6t00f16ou7eh7prv3b",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303301896910003,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303301896910003,
[junit4] 2> 2303300879474203,
[junit4] 2> 2303301896392503,
[junit4] 2> 2303300880739803,
[junit4] 2> 2303300880180203],
[junit4] 2> "_enqueue_time_":2303346507059053,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10073_solr",
[junit4] 2> "127.0.0.1:10016_solr",
[junit4] 2> "127.0.0.1:10049_solr",
[junit4] 2> "127.0.0.1:10058_solr",
[junit4] 2> "127.0.0.1:10083_solr"]}}
[junit4] 2> 1027232 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"82ed7436034b3T9pw511u6t00f16ou7eh7prv3b",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303301896910003,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303301896910003,
[junit4] 2> 2303300879474203,
[junit4] 2> 2303301896392503,
[junit4] 2> 2303300880739803,
[junit4] 2> 2303300880180203],
[junit4] 2> "_enqueue_time_":2303346507059053,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10073_solr",
[junit4] 2> "127.0.0.1:10016_solr",
[junit4] 2> "127.0.0.1:10049_solr",
[junit4] 2> "127.0.0.1:10058_solr",
[junit4] 2> "127.0.0.1:10083_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
[junit4] 2> 1027273 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node43
[junit4] 2> 1027306 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node58
[junit4] 2> 1027323 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10094_solr&inPlaceMove=true&replica=core_node19
[junit4] 2> 1027332 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10091_solr&inPlaceMove=true&replica=core_node39
[junit4] 2> 1027333 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2303351614 sessionWrapper.createTime 2303346650057453, this.sessionWrapper.createTime 2303346650057453
[junit4] 2> 1027333 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"82ed7436034b3T9pw511u6t00f16ou7eh7prv3b",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303301896910003,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303301896910003,
[junit4] 2> 2303300879474203,
[junit4] 2> 2303301896392503,
[junit4] 2> 2303300880739803,
[junit4] 2> 2303300880180203],
[junit4] 2> "_enqueue_time_":2303346507059053,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10073_solr",
[junit4] 2> "127.0.0.1:10016_solr",
[junit4] 2> "127.0.0.1:10049_solr",
[junit4] 2> "127.0.0.1:10058_solr",
[junit4] 2> "127.0.0.1:10083_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@52ece38f, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@108b7f38, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6ec83937, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@46b5d349], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
[junit4] 2> 1027333 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node43
[junit4] 2> 1027433 DEBUG (simCloudManagerPool-6293-thread-150) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node83":{
[junit4] 2> "core":"testBasic_shard3_replica_p82",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10053_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 1027545 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node58
[junit4] 2> 1027646 DEBUG (simCloudManagerPool-6293-thread-158) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node85":{
[junit4] 2> "core":"testBasic_shard4_replica_p84",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10037_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 1027758 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10094_solr&inPlaceMove=true&replica=core_node19
[junit4] 2> 1027858 DEBUG (simCloudManagerPool-6293-thread-167) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node87":{
[junit4] 2> "core":"testBasic_shard2_replica_n86",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10094_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1027971 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10091_solr&inPlaceMove=true&replica=core_node39
[junit4] 2> 1028071 DEBUG (simCloudManagerPool-6293-thread-179) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node89":{
[junit4] 2> "core":"testBasic_shard3_replica_t88",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10091_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 1028183 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 92694ms
[junit4] 2> 1028184 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.TestLargeCluster OP COUNTS: {CREATE=1, DELETESTATUS=7, MOVEREPLICA=7, REQUESTSTATUS=7, autoscaling=4, update=9}
[junit4] 2> 1028184 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 1028184 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 47598 ms for event id=82ed7436034b3T9pw511u6t00f16ou7eh7prv3b
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10043_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10081_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10084_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10036_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10041_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10014_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10011_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10038_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10067_solr
[junit4] 2> 1028284 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10042_solr
[junit4] 2> 1028304 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028324 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028345 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028365 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028385 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 82
[junit4] 2> 1028385 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10043_solr, 127.0.0.1:10067_solr, 127.0.0.1:10041_solr, 127.0.0.1:10042_solr, 127.0.0.1:10038_solr, 127.0.0.1:10081_solr, 127.0.0.1:10014_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10011_solr]
[junit4] 2> 1028385 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"82eede9cb89f9T9pw511u6t00f16ou7eh7prv3e",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303399178242553,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303399178242553,
[junit4] 2> 2303399181321553,
[junit4] 2> 2303399179841453,
[junit4] 2> 2303399181685453,
[junit4] 2> 2303399180955453,
[junit4] 2> 2303399178730503,
[junit4] 2> 2303399180214853,
[junit4] 2> 2303399179102403,
[junit4] 2> 2303399179472153,
[junit4] 2> 2303399180586853],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10067_solr",
[junit4] 2> "127.0.0.1:10041_solr",
[junit4] 2> "127.0.0.1:10042_solr",
[junit4] 2> "127.0.0.1:10038_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10014_solr",
[junit4] 2> "127.0.0.1:10084_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10011_solr"]}}
[junit4] 2> 1028385 DEBUG (ScheduledTrigger-6294-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
[junit4] 2> 1028386 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"82eede9cb89f9T9pw511u6t00f16ou7eh7prv3e",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303399178242553,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303399178242553,
[junit4] 2> 2303399181321553,
[junit4] 2> 2303399179841453,
[junit4] 2> 2303399181685453,
[junit4] 2> 2303399180955453,
[junit4] 2> 2303399178730503,
[junit4] 2> 2303399180214853,
[junit4] 2> 2303399179102403,
[junit4] 2> 2303399179472153,
[junit4] 2> 2303399180586853],
[junit4] 2> "_enqueue_time_":2303404251537753,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10067_solr",
[junit4] 2> "127.0.0.1:10041_solr",
[junit4] 2> "127.0.0.1:10042_solr",
[junit4] 2> "127.0.0.1:10038_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10014_solr",
[junit4] 2> "127.0.0.1:10084_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10011_solr"]}}
[junit4] 2> 1028386 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"82eede9cb89f9T9pw511u6t00f16ou7eh7prv3e",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303399178242553,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303399178242553,
[junit4] 2> 2303399181321553,
[junit4] 2> 2303399179841453,
[junit4] 2> 2303399181685453,
[junit4] 2> 2303399180955453,
[junit4] 2> 2303399178730503,
[junit4] 2> 2303399180214853,
[junit4] 2> 2303399179102403,
[junit4] 2> 2303399179472153,
[junit4] 2> 2303399180586853],
[junit4] 2> "_enqueue_time_":2303404251537753,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10067_solr",
[junit4] 2> "127.0.0.1:10041_solr",
[junit4] 2> "127.0.0.1:10042_solr",
[junit4] 2> "127.0.0.1:10038_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10014_solr",
[junit4] 2> "127.0.0.1:10084_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10011_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
[junit4] 2> 1028466 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10034_solr&inPlaceMove=true&replica=core_node28
[junit4] 2> 1028538 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10031_solr&inPlaceMove=true&replica=core_node48
[junit4] 2> 1028597 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10097_solr&inPlaceMove=true&replica=core_node2
[junit4] 2> 1028698 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10002_solr&inPlaceMove=true&replica=core_node6
[junit4] 2> 1028742 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10078_solr&inPlaceMove=true&replica=core_node17
[junit4] 2> 1028779 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10075_solr&inPlaceMove=true&replica=core_node34
[junit4] 2> 1028837 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10050_solr&inPlaceMove=true&replica=core_node4
[junit4] 2> 1028863 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10024_solr&inPlaceMove=true&replica=core_node62
[junit4] 2> 1028878 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10008_solr&inPlaceMove=true&replica=core_node68
[junit4] 2> 1028886 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10062_solr&inPlaceMove=true&replica=core_node24
[junit4] 2> 1028886 INFO (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2303429307 sessionWrapper.createTime 2303404344411903, this.sessionWrapper.createTime 2303404344411903
[junit4] 2> 1028887 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"82eede9cb89f9T9pw511u6t00f16ou7eh7prv3e",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2303399178242553,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 2303399178242553,
[junit4] 2> 2303399181321553,
[junit4] 2> 2303399179841453,
[junit4] 2> 2303399181685453,
[junit4] 2> 2303399180955453,
[junit4] 2> 2303399178730503,
[junit4] 2> 2303399180214853,
[junit4] 2> 2303399179102403,
[junit4] 2> 2303399179472153,
[junit4] 2> 2303399180586853],
[junit4] 2> "_enqueue_time_":2303404251537753,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10043_solr",
[junit4] 2> "127.0.0.1:10067_solr",
[junit4] 2> "127.0.0.1:10041_solr",
[junit4] 2> "127.0.0.1:10042_solr",
[junit4] 2> "127.0.0.1:10038_solr",
[junit4] 2> "127.0.0.1:10081_solr",
[junit4] 2> "127.0.0.1:10014_solr",
[junit4] 2> "127.0.0.1:10084_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10011_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@145c86ab, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2060a749, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2285b260, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@7fa88714, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@f313f94, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2ace9bd4, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@25573a5f, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@1efc99c6, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@1837d812, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@4679e0fb], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
[junit4] 2> 1028887 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10034_solr&inPlaceMove=true&replica=core_node28
[junit4] 2> 1028988 DEBUG (simCloudManagerPool-6293-thread-167) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node91":{
[junit4] 2> "core":"testBasic_shard2_replica_p90",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10034_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 1029099 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10031_solr&inPlaceMove=true&replica=core_node48
[junit4] 2> 1029200 DEBUG (simCloudManagerPool-6293-thread-195) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node93":{
[junit4] 2> "core":"testBasic_shard4_replica_n92",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10031_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1029311 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10097_solr&inPlaceMove=true&replica=core_node2
[junit4] 2> 1029412 DEBUG (simCloudManagerPool-6293-thread-44) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node95":{
[junit4] 2> "core":"testBasic_shard1_replica_n94",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10097_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1029524 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10002_solr&inPlaceMove=true&replica=core_node6
[junit4] 2> 1029624 DEBUG (simCloudManagerPool-6293-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node97":{
[junit4] 2> "core":"testBasic_shard1_replica_t96",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 1029735 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10078_solr&inPlaceMove=true&replica=core_node17
[junit4] 2> 1029835 DEBUG (simCloudManagerPool-6293-thread-110) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node99":{
[junit4] 2> "core":"testBasic_shard2_replica_n98",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10078_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1029947 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10075_solr&inPlaceMove=true&replica=core_node34
[junit4] 2> 1030047 DEBUG (simCloudManagerPool-6293-thread-172) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node101":{
[junit4] 2> "core":"testBasic_shard3_replica_n100",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10075_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1030158 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10050_solr&inPlaceMove=true&replica=core_node4
[junit4] 2> 1030258 DEBUG (simCloudManagerPool-6293-thread-155) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node103":{
[junit4] 2> "core":"testBasic_shard1_replica_n102",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10050_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1030370 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10024_solr&inPlaceMove=true&replica=core_node62
[junit4] 2> 1030470 DEBUG (simCloudManagerPool-6293-thread-129) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node105":{
[junit4] 2> "core":"testBasic_shard5_replica_n104",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10024_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 1030581 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10008_solr&inPlaceMove=true&replica=core_node68
[junit4] 2> 1030682 DEBUG (simCloudManagerPool-6293-thread-142) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node107":{
[junit4] 2> "core":"testBasic_shard5_replica_t106",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 1030793 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10062_solr&inPlaceMove=true&replica=core_node24
[junit4] 2> 1030894 DEBUG (simCloudManagerPool-6293-thread-92) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node109":{
[junit4] 2> "core":"testBasic_shard2_replica_t108",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10062_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 1031006 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 1031006 DEBUG (AutoscalingActionExecutor-6295-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 130995 ms for event id=82eede9cb89f9T9pw511u6t00f16ou7eh7prv3e
[junit4] 2> 1031023 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 21ms
[junit4] 2> 1031023 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.TestLargeCluster ==== Flaky replicas: 38. Additional MOVEREPLICA count: 10
[junit4] 2> 1031024 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBasic
[junit4] 2> 1031026 INFO (TEST-TestLargeCluster.testBasic-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 92
[junit4] 2> ## Empty nodes: 17
[junit4] 2> ## Dead nodes: 8
[junit4] 2> ## 127.0.0.1:10016_solr
[junit4] 2> ## 127.0.0.1:10026_solr
[junit4] 2> ## 127.0.0.1:10049_solr
[junit4] 2> ## 127.0.0.1:10058_solr
[junit4] 2> ## 127.0.0.1:10073_solr
[junit4] 2> ## 127.0.0.1:10077_solr
[junit4] 2> ## 127.0.0.1:10082_solr
[junit4] 2> ## 127.0.0.1:10083_solr
[junit4] 2> ## Collections: [.system, testBasic]
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 76
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ## * testBasic 75
[junit4] 2> ## - active 75
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETESTATUS 17
[junit4] 2> ## - MOVEREPLICA 17
[junit4] 2> ## - REQUESTSTATUS 17
[junit4] 2> ## - autoscaling 4
[junit4] 2> ## - update 20
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_trigger1
[junit4] 2> ## - STARTED 3
[junit4] 2> ## - SUCCEEDED 3
[junit4] 2>
[junit4] 2> 1031074 INFO (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10011_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10082_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10014_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10049_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10036_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10016_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10038_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10077_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10041_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10083_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10042_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10026_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10043_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10073_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10067_solr
[junit4] 2> 1031074 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10058_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10081_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeLost/127.0.0.1:10084_solr
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1031074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1031075 INFO (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1031075 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1031075 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1031077 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1031077 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1031078 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion -1
[junit4] 2> 1031078 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1031078 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10036_solr, 127.0.0.1:10084_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10070_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10087_solr, 127.0.0.1:10041_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 1031079 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1031079 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1031079 DEBUG (ScheduledTrigger-6297-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031100 DEBUG (ScheduledTrigger-6297-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031120 DEBUG (ScheduledTrigger-6297-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031140 DEBUG (ScheduledTrigger-6297-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031161 DEBUG (ScheduledTrigger-6297-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031179 DEBUG (simCloudManagerPool-6296-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1031179 DEBUG (simCloudManagerPool-6296-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1031179 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1031180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1031180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1031181 DEBUG (ScheduledTrigger-6297-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 1031181 DEBUG (simCloudManagerPool-6296-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1031181 DEBUG (simCloudManagerPool-6296-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1031181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1031182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1031182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1031182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1031204 DEBUG (simCloudManagerPool-6296-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1031204 DEBUG (simCloudManagerPool-6296-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1031204 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 1031205 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger3 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10036_solr, 127.0.0.1:10084_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10070_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_sol
[...truncated too long message...]
\"node_name\":\"127.0.0.1:10085_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"SEARCHER.searcher.maxDoc\":0,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":10240,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10063_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4,\n \"INDEX.sizeInGB\":9.5367431640625E-6,\n \"SEARCHER.searcher.numDocs\":0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "__start__":3,
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "hotCollections":{"testSearchRate":40.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10001_solr":4.0,
[junit4] 2> "127.0.0.1:10085_solr":4.0,
[junit4] 2> "127.0.0.1:10090_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10063_solr":4.0,
[junit4] 2> "127.0.0.1:10092_solr":4.0,
[junit4] 2> "127.0.0.1:10093_solr":4.0,
[junit4] 2> "127.0.0.1:10060_solr":4.0,
[junit4] 2> "127.0.0.1:10023_solr":4.0},
[junit4] 2> "_enqueue_time_":2316479535593604,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@376838a7, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@22d1e510, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@436ae447], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 222815 DEBUG (AutoscalingActionExecutor-18-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> 222836 INFO (simCloudManagerPool-16-thread-31) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10056_solr for creating new replica
[junit4] 2> 222870 DEBUG (AutoscalingActionExecutor-18-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10059_solr&type=NRT
[junit4] 2> 222878 INFO (simCloudManagerPool-16-thread-36) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10059_solr for creating new replica
[junit4] 2> 222894 DEBUG (AutoscalingActionExecutor-18-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10053_solr&type=NRT
[junit4] 2> 222894 INFO (simCloudManagerPool-16-thread-41) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10053_solr for creating new replica
[junit4] 2> 222908 DEBUG (simCloudManagerPool-16-thread-46) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node8":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10092_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 222908 DEBUG (AutoscalingActionExecutor-18-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: search_rate_trigger after 100ms
[junit4] 2> 222908 DEBUG (AutoscalingActionExecutor-18-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 5781 ms for event id=83ad1fa2c1de2T9pw511u6t00f16ou7eh7prpkr
[junit4] 2> 222947 INFO (TEST-TestLargeCluster.testSearchRate-seed#[1961A9D5677685CF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 222948 INFO (TEST-TestLargeCluster.testSearchRate-seed#[1961A9D5677685CF]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 100
[junit4] 2> ## Empty nodes: 77
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system, testSearchRate]
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 24
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ## * testSearchRate 23
[junit4] 2> ## - active 23
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - ADDREPLICA 3
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETESTATUS 3
[junit4] 2> ## - REQUESTSTATUS 3
[junit4] 2> ## - autoscaling 4
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: search_rate_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] OK 1.74s J1 | TestLargeCluster.testSearchRate
[junit4] 2> 222949 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 222949 DEBUG (SUITE-TestLargeCluster-seed#[1961A9D5677685CF]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_1961A9D5677685CF-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=dua, timezone=Asia/Qatar
[junit4] 2> NOTE: Linux 4.13.0-41-generic amd64/Oracle Corporation 10.0.1 (64-bit)/cpus=8,threads=1,free=312319040,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestLargeCluster]
[junit4] Completed [3/5 (3!)] on J1 in 222.18s, 4 tests, 1 failure <<< FAILURES!
[...truncated 27 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 20 tests, 3 failures [seed: 1961A9D5677685CF]
Total time: 4 minutes 16 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 3/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=35a7e95bce54d53820ce3e0b1e2966f609a1c1d2, workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #50
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2