You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/11/06 01:42:27 UTC

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4914 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4914/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseParallelGC

6 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica

Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/49)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"1",   "tlogReplicas":"0",   "autoCreated":"true",   "policy":"c1",   "shards":{"shard1":{       "replicas":{"core_node1":{           "core":"testCreateCollectionAddReplica_shard1_replica_n1",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10003_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"80000000-7fffffff",       "state":"active"}}}

Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/49)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10003_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}
	at __randomizedtesting.SeedInfo.seed([8380A4AA77D96278:3A0C184669A8ADE]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
	at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 50
live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/49)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10003_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/49)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10003_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
	... 40 more


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist

Error Message:
Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:57791_solr, 127.0.0.1:57789_solr, 127.0.0.1:57790_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).

Stack Trace:
java.lang.RuntimeException: Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:57791_solr, 127.0.0.1:57789_solr, 127.0.0.1:57790_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).
	at __randomizedtesting.SeedInfo.seed([B26ED4BE3BB82EB5:D0CDCA10393A4AA5]:0)
	at org.apache.solr.client.solrj.impl.HttpClusterStateProvider.getState(HttpClusterStateProvider.java:108)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.resolveAliases(CloudSolrClient.java:1115)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:844)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist(CloudSolrClientTest.java:779)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist

Error Message:
Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58042_solr, 127.0.0.1:58043_solr, 127.0.0.1:58044_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).

Stack Trace:
java.lang.RuntimeException: Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58042_solr, 127.0.0.1:58043_solr, 127.0.0.1:58044_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).
	at __randomizedtesting.SeedInfo.seed([B26ED4BE3BB82EB5:D0CDCA10393A4AA5]:0)
	at org.apache.solr.client.solrj.impl.HttpClusterStateProvider.getState(HttpClusterStateProvider.java:108)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.resolveAliases(CloudSolrClient.java:1115)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:844)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist(CloudSolrClientTest.java:779)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist

Error Message:
Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58029_solr, 127.0.0.1:58030_solr, 127.0.0.1:58028_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).

Stack Trace:
java.lang.RuntimeException: Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58029_solr, 127.0.0.1:58030_solr, 127.0.0.1:58028_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).
	at __randomizedtesting.SeedInfo.seed([B26ED4BE3BB82EB5:D0CDCA10393A4AA5]:0)
	at org.apache.solr.client.solrj.impl.HttpClusterStateProvider.getState(HttpClusterStateProvider.java:108)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.resolveAliases(CloudSolrClient.java:1115)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:844)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist(CloudSolrClientTest.java:779)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist

Error Message:
Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58279_solr, 127.0.0.1:58278_solr, 127.0.0.1:58277_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).

Stack Trace:
java.lang.RuntimeException: Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:58279_solr, 127.0.0.1:58278_solr, 127.0.0.1:58277_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).
	at __randomizedtesting.SeedInfo.seed([B26ED4BE3BB82EB5:D0CDCA10393A4AA5]:0)
	at org.apache.solr.client.solrj.impl.HttpClusterStateProvider.getState(HttpClusterStateProvider.java:108)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.resolveAliases(CloudSolrClient.java:1115)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:844)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist(CloudSolrClientTest.java:779)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist

Error Message:
Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:55005_solr, 127.0.0.1:55006_solr, 127.0.0.1:55004_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).

Stack Trace:
java.lang.RuntimeException: Tried fetching cluster state using the node names we knew of, i.e. [127.0.0.1:55005_solr, 127.0.0.1:55006_solr, 127.0.0.1:55004_solr]. However, succeeded in obtaining the cluster state from none of them.If you think your Solr cluster is up and is accessible, you could try re-creating a new CloudSolrClient using working solrUrl(s) or zkHost(s).
	at __randomizedtesting.SeedInfo.seed([B26ED4BE3BB82EB5:D0CDCA10393A4AA5]:0)
	at org.apache.solr.client.solrj.impl.HttpClusterStateProvider.getState(HttpClusterStateProvider.java:108)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.resolveAliases(CloudSolrClient.java:1115)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:844)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
	at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testCollectionDoesntExist(CloudSolrClientTest.java:779)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13009 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_8380A4AA77D96278-001/init-core-data-001
   [junit4]   2> 525513 WARN  (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 525513 INFO  (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 525514 INFO  (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 525519 INFO  (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 525556 DEBUG (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 525557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 525557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 525557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 525557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 525558 DEBUG (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
   [junit4]   2> 525561 DEBUG (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 525561 INFO  (SUITE-TestSimPolicyCloud-seed#[8380A4AA77D96278]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 525562 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 525562 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 525562 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4] IGNOR/A 0.00s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4]    > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028"))
   [junit4]   2> 525565 DEBUG (ScheduledTrigger-3045-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525571 INFO  (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDataProviderPerReplicaDetails
   [junit4]   2> 525571 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 525572 INFO  (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 525572 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 525573 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 525574 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 525575 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 525575 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 525575 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 525576 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 525577 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 525577 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 525577 DEBUG (ScheduledTrigger-3050-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525609 DEBUG (ScheduledTrigger-3050-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525630 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525658 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525687 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525712 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525733 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525757 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525778 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525778 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection perReplicaDataColl, currentVersion=1
   [junit4]   2> 525778 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
   [junit4]   2> 525778 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 525792 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
   [junit4]   2> 525795 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 525801 DEBUG (simCloudManagerPool-3049-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection perReplicaDataColl, currentVersion=3
   [junit4]   2> 525801 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525814 DEBUG (simCloudManagerPool-3049-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
   [junit4]   2> 525815 DEBUG (simCloudManagerPool-3049-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 525819 DEBUG (simCloudManagerPool-3049-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for perReplicaDataColl / shard1 (currentVersion=4): {"core_node1":{
   [junit4]   2>     "core":"perReplicaDataColl_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"perReplicaDataColl",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 525823 DEBUG (ScheduledTrigger-3050-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525828 DEBUG (simCloudManagerPool-3049-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
   [junit4]   2> 525828 DEBUG (simCloudManagerPool-3049-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 525842 DEBUG (simCloudManagerPool-3049-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
   [junit4]   2> 525843 DEBUG (simCloudManagerPool-3049-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 525852 DEBUG (ScheduledTrigger-3050-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525853 DEBUG (simCloudManagerPool-3049-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
   [junit4]   2> 525854 DEBUG (simCloudManagerPool-3049-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 525866 DEBUG (simCloudManagerPool-3049-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
   [junit4]   2> 525867 DEBUG (simCloudManagerPool-3049-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
   [junit4]   2> 525867 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
   [junit4]   2> 525867 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
   [junit4]   2> 525873 DEBUG (ScheduledTrigger-3050-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525903 DEBUG (ScheduledTrigger-3050-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525929 DEBUG (ScheduledTrigger-3050-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525949 DEBUG (ScheduledTrigger-3050-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525979 DEBUG (ScheduledTrigger-3050-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 525980 INFO  (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDataProviderPerReplicaDetails
   [junit4]   2> 525981 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
   [junit4]   2> 525981 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
   [junit4]   2> 525982 INFO  (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		5
   [junit4]   2> ## Empty nodes:	5
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:
   [junit4]   2> ## Max replicas per node:	0
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		0
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- DELETE             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> 
   [junit4]   2> 525986 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSplitShard
   [junit4]   2> 525986 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 525986 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 525986 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 525986 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 525987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 525987 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 525987 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 525988 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 525989 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 525989 DEBUG (ScheduledTrigger-3053-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526018 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526045 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526075 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526102 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526123 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526146 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526174 DEBUG (ScheduledTrigger-3053-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526198 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526199 DEBUG (simCloudManagerPool-3052-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
   [junit4]   2> 526199 DEBUG (simCloudManagerPool-3052-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
   [junit4]   2> 526200 DEBUG (simCloudManagerPool-3052-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 526200 DEBUG (simCloudManagerPool-3052-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 526201 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 526201 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 526201 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 526201 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionSplitShard, currentVersion=11
   [junit4]   2> 526202 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
   [junit4]   2> 526202 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
   [junit4]   2> 526217 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
   [junit4]   2> 526217 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
   [junit4]   2> 526219 DEBUG (ScheduledTrigger-3053-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526219 DEBUG (simCloudManagerPool-3052-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionSplitShard, currentVersion=13
   [junit4]   2> 526235 DEBUG (simCloudManagerPool-3052-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
   [junit4]   2> 526235 DEBUG (simCloudManagerPool-3052-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
   [junit4]   2> 526235 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
   [junit4]   2> 526236 DEBUG (simCloudManagerPool-3052-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1 (currentVersion=14): {"core_node1":{
   [junit4]   2>     "core":"testCreateCollectionSplitShard_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCreateCollectionSplitShard",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 526236 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
   [junit4]   2> 526249 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526249 DEBUG (simCloudManagerPool-3052-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
   [junit4]   2> 526250 DEBUG (simCloudManagerPool-3052-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
   [junit4]   2> 526279 DEBUG (ScheduledTrigger-3053-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526309 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526331 DEBUG (ScheduledTrigger-3053-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526362 INFO  (simCloudManagerPool-3052-thread-6) [    ] o.a.s.c.a.c.SplitShardCmd numSubShards set at: 2
   [junit4]   2> 526362 DEBUG (ScheduledTrigger-3053-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526392 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526422 DEBUG (simCloudManagerPool-3052-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
   [junit4]   2> 526422 DEBUG (ScheduledTrigger-3053-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526422 DEBUG (simCloudManagerPool-3052-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
   [junit4]   2> 526424 DEBUG (simCloudManagerPool-3052-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_1 (currentVersion=17): {"core_node10":{
   [junit4]   2>     "core":"testCreateCollectionSplitShard_shard1_1_replica_n9",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"testCreateCollectionSplitShard",
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 526424 DEBUG (simCloudManagerPool-3052-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_0 (currentVersion=17): {"core_node6":{
   [junit4]   2>     "core":"testCreateCollectionSplitShard_shard1_0_replica_n5",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"testCreateCollectionSplitShard",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":20480,
   [junit4]   2>     "base_url":"http://127.0.0.1:10003/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":1.9073486328125E-5,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 526424 DEBUG (simCloudManagerPool-3052-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
   [junit4]   2> 526424 DEBUG (simCloudManagerPool-3052-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
   [junit4]   2> 526426 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
   [junit4]   2> 526426 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
   [junit4]   2> 526445 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526475 DEBUG (ScheduledTrigger-3053-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526495 DEBUG (ScheduledTrigger-3053-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526516 DEBUG (ScheduledTrigger-3053-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526541 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSplitShard
   [junit4]   2> 526541 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
   [junit4]   2> 526541 DEBUG (ScheduledTrigger-3053-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526541 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
   [junit4]   2> 526542 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		5
   [junit4]   2> ## Empty nodes:	5
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:
   [junit4]   2> ## Max replicas per node:	0
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		0
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- DELETE             1
   [junit4]   2> ##		- SPLITSHARD         1
   [junit4]   2> ##		- autoscaling        1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> 
   [junit4]   2> 526549 INFO  (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDataProvider
   [junit4]   2> 526549 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 526549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 526549 INFO  (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 526549 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 526550 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 526550 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 526550 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 526551 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 526552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 526552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 526552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 526552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 526553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 526553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 526553 DEBUG (ScheduledTrigger-3056-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526578 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526599 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526621 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526642 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526662 DEBUG (ScheduledTrigger-3056-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526683 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526703 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526726 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526747 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526753 DEBUG (simCloudManagerPool-3055-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection policiesTest, currentVersion=20
   [junit4]   2> 526753 DEBUG (simCloudManagerPool-3055-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
   [junit4]   2> 526753 DEBUG (simCloudManagerPool-3055-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
   [junit4]   2> 526758 DEBUG (MetricsHistoryHandler-3048-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
   [junit4]   2> 526759 DEBUG (MetricsHistoryHandler-3048-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
   [junit4]   2> 526765 DEBUG (simCloudManagerPool-3055-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection policiesTest, currentVersion=22
   [junit4]   2> 526769 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526776 DEBUG (simCloudManagerPool-3055-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
   [junit4]   2> 526776 DEBUG (simCloudManagerPool-3055-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
   [junit4]   2> 526776 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
   [junit4]   2> 526777 DEBUG (simCloudManagerPool-3055-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / shard1 (currentVersion=23): {"core_node1":{
   [junit4]   2>     "core":"policiesTest_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"policiesTest",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 526777 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
   [junit4]   2> 526790 DEBUG (simCloudManagerPool-3055-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
   [junit4]   2> 526790 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526790 DEBUG (simCloudManagerPool-3055-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
   [junit4]   2> 526812 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526825 INFO  (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDataProvider
   [junit4]   2> 526826 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=25
   [junit4]   2> 526826 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 25
   [junit4]   2> 526835 DEBUG (ScheduledTrigger-3056-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526855 INFO  (TEST-TestSimPolicyCloud.testDataProvider-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		5
   [junit4]   2> ## Empty nodes:	3
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:
   [junit4]   2> ##  * policiesTest
   [junit4]   2> ##    shardsTotal	1
   [junit4]   2> ##    shardsState	{active=1}
   [junit4]   2> ##      shardsWithoutLeader	0
   [junit4]   2> ##    totalReplicas	2
   [junit4]   2> ##      activeReplicas	2
   [junit4]   2> ##      inactiveReplicas	0
   [junit4]   2> ##    totalActiveDocs	0
   [junit4]   2> ##      bufferedDocs	0
   [junit4]   2> ##      maxActiveSliceDocs	0
   [junit4]   2> ##      minActiveSliceDocs	0
   [junit4]   2> ##      avgActiveSliceDocs	0
   [junit4]   2> ##    totalInactiveDocs	0
   [junit4]   2> ##      maxInactiveSliceDocs	0
   [junit4]   2> ##      minInactiveSliceDocs	0
   [junit4]   2> ##      avgInactiveSliceDocs	NaN
   [junit4]   2> ##    totalActiveBytes	10,240
   [junit4]   2> ##      maxActiveSliceBytes	10,240
   [junit4]   2> ##      minActiveSliceBytes	10,240
   [junit4]   2> ##      avgActiveSliceBytes	10,240
   [junit4]   2> ##    totalInactiveBytes	0
   [junit4]   2> ##      maxInactiveSliceBytes	0
   [junit4]   2> ##      minInactiveSliceBytes	0
   [junit4]   2> ##      avgInactiveSliceBytes	NaN
   [junit4]   2> ##    totalActiveDeletedDocs	0
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		2
   [junit4]   2> ## * policiesTest		2
   [junit4]   2> ##		- active           2
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- ADDROLE            1
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> 
   [junit4]   2> 526856 DEBUG (ScheduledTrigger-3056-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526859 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4]   2> 526859 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 526859 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 526859 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 526860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 526860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 526860 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 526860 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 526860 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 526860 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 526860 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 526861 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 526861 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 526862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 526862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 526862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
   [junit4]   2> 526862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 526863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 526863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 526863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 526863 DEBUG (ScheduledTrigger-3059-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526894 DEBUG (ScheduledTrigger-3059-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526923 DEBUG (ScheduledTrigger-3059-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526953 DEBUG (ScheduledTrigger-3059-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 526983 DEBUG (ScheduledTrigger-3059-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527010 DEBUG (ScheduledTrigger-3059-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527034 DEBUG (ScheduledTrigger-3059-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527064 DEBUG (ScheduledTrigger-3059-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527064 INFO  (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.TestSimPolicyCloud NRT 127.0.0.1:10001_solr PULL 127.0.0.1:10004_solr , TLOG 127.0.0.1:10000_solr 
   [junit4]   2> 527065 DEBUG (simCloudManagerPool-3058-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=26
   [junit4]   2> 527065 DEBUG (simCloudManagerPool-3058-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 26
   [junit4]   2> 527066 DEBUG (simCloudManagerPool-3058-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 527066 DEBUG (simCloudManagerPool-3058-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 527066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 527066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 527066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 527067 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection policiesTest, currentVersion=27
   [junit4]   2> 527068 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=27
   [junit4]   2> 527068 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 27
   [junit4]   2> 527079 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=28
   [junit4]   2> 527079 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 28
   [junit4]   2> 527082 DEBUG (simCloudManagerPool-3058-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection policiesTest, currentVersion=29
   [junit4]   2> 527092 DEBUG (ScheduledTrigger-3059-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527092 DEBUG (simCloudManagerPool-3058-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=29
   [junit4]   2> 527092 DEBUG (simCloudManagerPool-3058-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 29
   [junit4]   2> 527093 DEBUG (simCloudManagerPool-3058-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / s1 (currentVersion=30): {"core_node3":{
   [junit4]   2>     "core":"policiesTest_s1_replica_n1",
   [junit4]   2>     "shard":"s1",
   [junit4]   2>     "collection":"policiesTest",
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 527108 DEBUG (simCloudManagerPool-3058-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=30
   [junit4]   2> 527108 DEBUG (simCloudManagerPool-3058-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 30
   [junit4]   2> 527119 DEBUG (simCloudManagerPool-3058-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=31
   [junit4]   2> 527119 DEBUG (ScheduledTrigger-3059-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527120 DEBUG (simCloudManagerPool-3058-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 31
   [junit4]   2> 527120 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=32
   [junit4]   2> 527120 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[8380A4AA77D96278]) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 32
   [junit4]   2> 527145 DEBUG (ScheduledTrigger-3059-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527173 DEBUG (ScheduledTrigger-3059-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527199 DEBUG (ScheduledTrigger-3059-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527227 DEBUG (ScheduledTrigger-3059-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 527231 INFO  (simCloudManagerPool-3058-thread-7) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica of shard s3 for collection policiesTest
   [junit4]   2> 527246 DEBUG (simCloudManagerPool-3058-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=33
   [junit4]   2> 527246 DEBUG (simCloudManagerPool-3058-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 33
   [junit4]   2> 527246 INFO  (simCloudManagerPool-3058-thread-7) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica of shard s3 for collection policiesTest
   [junit4]   2> 527247 DEBUG (simCloudManagerPool-3058-thread-8) [    ] o.a.s.c.a.s.SimClus

[...truncated too long message...]

cene-Solr-master-MacOSX/solr/build/solr-core/classes/test
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/classes/test/org/apache/solr/cloud/autoscaling/sim/package-info.class
     [copy] Copying 1 file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/classes/test

common-solr.compile-test:

compile-test:

BUILD SUCCESSFUL
Total time: 35 seconds
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSimPolicyCloud" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=8380A4AA77D96278 -Dtests.slow=true -Dtests.locale=eo-001 -Dtests.timezone=AST -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
Buildfile: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build.xml

-clover.disable:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

install-junit4-taskdef:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.15 in public
[ivy:cachepath] :: resolution report :: resolve 12ms :: artifacts dl 0ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-init-totals:

-test:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test
[junit4:pickseed] Seed property 'tests.seed' already defined: 8380A4AA77D96278
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/temp
   [junit4] <JUnit4> says salut! Master seed: 8380A4AA77D96278
   [junit4] Executing 5 suites with 2 JVMs.
   [junit4] 
   [junit4] Started J1 PID(8702@serv1-vm2.sd-datasolutions.de).
   [junit4] Started J0 PID(8701@serv1-vm2.sd-datasolutions.de).
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4] OK      0.71s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4] OK      0.42s J1 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
   [junit4] OK      0.54s J1 | TestSimPolicyCloud.testCreateCollectionSplitShard
   [junit4] OK      0.30s J1 | TestSimPolicyCloud.testDataProvider
   [junit4] OK      0.46s J1 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4] OK      0.27s J1 | TestSimPolicyCloud.testMetricsTag
   [junit4] OK      0.28s J1 | TestSimPolicyCloud.testCreateCollectionAddReplica
   [junit4] Completed [1/5] on J1 in 4.98s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4] OK      0.72s J0 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4] OK      0.42s J0 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
   [junit4] OK      0.55s J0 | TestSimPolicyCloud.testCreateCollectionSplitShard
   [junit4] OK      0.30s J0 | TestSimPolicyCloud.testDataProvider
   [junit4] OK      0.43s J0 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4] OK      0.27s J0 | TestSimPolicyCloud.testMetricsTag
   [junit4] OK      0.28s J0 | TestSimPolicyCloud.testCreateCollectionAddReplica
   [junit4] Completed [2/5] on J0 in 5.00s, 7 tests
   [junit4] 
   [junit4] Duplicate suite name used with XML reports: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4] OK      0.41s J0 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4] OK      0.42s J0 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
   [junit4] OK      0.55s J0 | TestSimPolicyCloud.testCreateCollectionSplitShard
   [junit4] OK      0.31s J0 | TestSimPolicyCloud.testDataProvider
   [junit4] OK      0.44s J0 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4] OK      0.26s J0 | TestSimPolicyCloud.testMetricsTag
   [junit4] OK      0.28s J0 | TestSimPolicyCloud.testCreateCollectionAddReplica
   [junit4] Completed [3/5] on J0 in 2.69s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4] OK      0.39s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4] OK      0.43s J1 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
   [junit4] OK      0.55s J1 | TestSimPolicyCloud.testCreateCollectionSplitShard
   [junit4] OK      0.30s J1 | TestSimPolicyCloud.testDataProvider
   [junit4] OK      0.43s J1 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4] OK      0.26s J1 | TestSimPolicyCloud.testMetricsTag
   [junit4] OK      0.26s J1 | TestSimPolicyCloud.testCreateCollectionAddReplica
   [junit4] Completed [4/5] on J1 in 2.64s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
   [junit4] OK      0.38s J0 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
   [junit4] OK      0.43s J0 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
   [junit4] OK      0.55s J0 | TestSimPolicyCloud.testCreateCollectionSplitShard
   [junit4] OK      0.30s J0 | TestSimPolicyCloud.testDataProvider
   [junit4] OK      0.44s J0 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
   [junit4] OK      0.26s J0 | TestSimPolicyCloud.testMetricsTag
   [junit4] OK      0.28s J0 | TestSimPolicyCloud.testCreateCollectionAddReplica
   [junit4] Completed [5/5] on J0 in 2.65s, 7 tests
   [junit4] 
   [junit4] JVM J0:     1.14 ..    12.53 =    11.40s
   [junit4] JVM J1:     0.91 ..     9.91 =     9.00s
   [junit4] Execution time total: 12 seconds
   [junit4] Tests summary: 5 suites, 35 tests
   [junit4] Could not remove temporary path: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp])
   [junit4] Could not remove temporary path: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp])
     [echo] 5 slowest tests:
[junit4:tophints] 517.93s | org.apache.solr.cloud.cdcr.CdcrWithNodesRestartsTest
[junit4:tophints] 319.05s | org.apache.solr.cloud.cdcr.CdcrOpsAndBoundariesTest
[junit4:tophints] 315.94s | org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4:tophints] 167.88s | org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4:tophints] 117.81s | org.apache.solr.update.SoftAutoCommitTest

-check-totals:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/Users/jenkins/.ivy2/cache/org.codehaus.groovy/groovy-all/jars/groovy-all-2.4.15.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

test-nocompile:

BUILD SUCCESSFUL
Total time: 15 seconds
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[repro]   5/5 failed: org.apache.solr.client.solrj.impl.CloudSolrClientTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=be65b95e80fdddea109a9d850506d6c524911ecb, workspace=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX
[WARNINGS] Computing warning deltas based on reference build #4913
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4915 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4915/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

11 tests failed.
FAILED:  org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica

Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:62934_solr, 127.0.0.1:62935_solr, 127.0.0.1:62936_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"raceDeleteReplica_false_shard1_replica_n1",           "base_url":"http://127.0.0.1:62937/solr",           "node_name":"127.0.0.1:62937_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node6":{           "core":"raceDeleteReplica_false_shard1_replica_n5",           "base_url":"http://127.0.0.1:62937/solr",           "node_name":"127.0.0.1:62937_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:62934_solr, 127.0.0.1:62935_solr, 127.0.0.1:62936_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"raceDeleteReplica_false_shard1_replica_n1",
          "base_url":"http://127.0.0.1:62937/solr",
          "node_name":"127.0.0.1:62937_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node6":{
          "core":"raceDeleteReplica_false_shard1_replica_n5",
          "base_url":"http://127.0.0.1:62937/solr",
          "node_name":"127.0.0.1:62937_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:3FEAB75D2510B4E7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest

Error Message:
Error from server at http://127.0.0.1:62934/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62934/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:F89C628650DD5658]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards

Error Message:
Error from server at http://127.0.0.1:62934/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62934/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:696D91D902A3A378]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards(DeleteReplicaTest.java:161)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica

Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:63239_solr, 127.0.0.1:63240_solr, 127.0.0.1:63241_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"raceDeleteReplica_false_shard1_replica_n1",           "base_url":"http://127.0.0.1:63242/solr",           "node_name":"127.0.0.1:63242_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node6":{           "core":"raceDeleteReplica_false_shard1_replica_n5",           "base_url":"http://127.0.0.1:63242/solr",           "node_name":"127.0.0.1:63242_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:63239_solr, 127.0.0.1:63240_solr, 127.0.0.1:63241_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"raceDeleteReplica_false_shard1_replica_n1",
          "base_url":"http://127.0.0.1:63242/solr",
          "node_name":"127.0.0.1:63242_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node6":{
          "core":"raceDeleteReplica_false_shard1_replica_n5",
          "base_url":"http://127.0.0.1:63242/solr",
          "node_name":"127.0.0.1:63242_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:3FEAB75D2510B4E7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest

Error Message:
Error from server at http://127.0.0.1:63241/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:63241/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:F89C628650DD5658]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards

Error Message:
Error from server at http://127.0.0.1:63241/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:63241/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:696D91D902A3A378]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards(DeleteReplicaTest.java:161)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica

Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:63993_solr, 127.0.0.1:63994_solr, 127.0.0.1:63995_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"raceDeleteReplica_false_shard1_replica_n1",           "base_url":"http://127.0.0.1:63996/solr",           "node_name":"127.0.0.1:63996_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node6":{           "core":"raceDeleteReplica_false_shard1_replica_n5",           "base_url":"http://127.0.0.1:63996/solr",           "node_name":"127.0.0.1:63996_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:63993_solr, 127.0.0.1:63994_solr, 127.0.0.1:63995_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"raceDeleteReplica_false_shard1_replica_n1",
          "base_url":"http://127.0.0.1:63996/solr",
          "node_name":"127.0.0.1:63996_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node6":{
          "core":"raceDeleteReplica_false_shard1_replica_n5",
          "base_url":"http://127.0.0.1:63996/solr",
          "node_name":"127.0.0.1:63996_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:3FEAB75D2510B4E7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest

Error Message:
Error from server at http://127.0.0.1:63993/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:63993/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:F89C628650DD5658]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica

Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:62323_solr, 127.0.0.1:62324_solr, 127.0.0.1:62325_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"raceDeleteReplica_false_shard1_replica_n1",           "base_url":"http://127.0.0.1:62326/solr",           "node_name":"127.0.0.1:62326_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node6":{           "core":"raceDeleteReplica_false_shard1_replica_n5",           "base_url":"http://127.0.0.1:62326/solr",           "node_name":"127.0.0.1:62326_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:62323_solr, 127.0.0.1:62324_solr, 127.0.0.1:62325_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"raceDeleteReplica_false_shard1_replica_n1",
          "base_url":"http://127.0.0.1:62326/solr",
          "node_name":"127.0.0.1:62326_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node6":{
          "core":"raceDeleteReplica_false_shard1_replica_n5",
          "base_url":"http://127.0.0.1:62326/solr",
          "node_name":"127.0.0.1:62326_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:3FEAB75D2510B4E7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
	at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest

Error Message:
Error from server at http://127.0.0.1:62323/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62323/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:F89C628650DD5658]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards

Error Message:
Error from server at http://127.0.0.1:62323/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62323/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
	at __randomizedtesting.SeedInfo.seed([55FCD68D4DE2FE2D:696D91D902A3A378]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards(DeleteReplicaTest.java:161)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12788 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> 554359 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-001/init-core-data-001
   [junit4]   2> 554360 WARN  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 554360 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 554365 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 554365 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-001/tempDir-001
   [junit4]   2> 554366 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 554366 INFO  (Thread-701) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 554366 INFO  (Thread-701) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 554369 ERROR (Thread-701) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 554467 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:62321
   [junit4]   2> 554477 INFO  (zkConnectionManagerCallback-1207-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554484 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 554484 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 554484 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 554485 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 554486 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 554486 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 554486 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 554487 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67a25456{/solr,null,AVAILABLE}
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37fa1321{/solr,null,AVAILABLE}
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@48ca726e{HTTP/1.1,[http/1.1]}{127.0.0.1:62323}
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@59979f57{HTTP/1.1,[http/1.1]}{127.0.0.1:62324}
   [junit4]   2> 554488 INFO  (jetty-launcher-1204-thread-3) [    ] o.e.j.s.Server Started @554572ms
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.e.j.s.Server Started @554572ms
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62323}
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62324}
   [junit4]   2> 554489 ERROR (jetty-launcher-1204-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 554489 ERROR (jetty-launcher-1204-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-06T17:14:54.437Z
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 554489 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-06T17:14:54.437Z
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63b65b24{/solr,null,AVAILABLE}
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@743933f3{/solr,null,AVAILABLE}
   [junit4]   2> 554490 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@76487a83{HTTP/1.1,[http/1.1]}{127.0.0.1:62325}
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.e.j.s.Server Started @554574ms
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62325}
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@40d70873{HTTP/1.1,[http/1.1]}{127.0.0.1:62326}
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.e.j.s.Server Started @554574ms
   [junit4]   2> 554491 ERROR (jetty-launcher-1204-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62326}
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 554491 ERROR (jetty-launcher-1204-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-06T17:14:54.439Z
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 554491 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-06T17:14:54.439Z
   [junit4]   2> 554495 INFO  (zkConnectionManagerCallback-1209-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554495 INFO  (zkConnectionManagerCallback-1215-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554496 INFO  (zkConnectionManagerCallback-1211-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554496 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 554496 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 554497 INFO  (zkConnectionManagerCallback-1213-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554497 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 554502 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 554603 INFO  (jetty-launcher-1204-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62321/solr
   [junit4]   2> 554605 INFO  (zkConnectionManagerCallback-1219-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554608 INFO  (zkConnectionManagerCallback-1221-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554687 INFO  (jetty-launcher-1204-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62321/solr
   [junit4]   2> 554689 INFO  (zkConnectionManagerCallback-1227-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554692 INFO  (zkConnectionManagerCallback-1229-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554718 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:62323_solr
   [junit4]   2> 554719 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.Overseer Overseer (id=72065543657357320-127.0.0.1:62323_solr-n_0000000000) starting
   [junit4]   2> 554730 INFO  (zkConnectionManagerCallback-1236-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554731 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62324_solr
   [junit4]   2> 554733 INFO  (zkCallback-1228-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 554744 INFO  (zkCallback-1220-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 554766 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 554768 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:62321/solr ready
   [junit4]   2> 554769 INFO  (OverseerStateUpdate-72065543657357320-127.0.0.1:62323_solr-n_0000000000) [n:127.0.0.1:62323_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:62323_solr
   [junit4]   2> 554773 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 554773 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62323_solr
   [junit4]   2> 554773 DEBUG (OverseerStateUpdate-72065543657357320-127.0.0.1:62323_solr-n_0000000000) [n:127.0.0.1:62323_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:62323_solr"} current state version: 0
   [junit4]   2> 554775 INFO  (zkCallback-1228-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 554776 INFO  (zkCallback-1235-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 554776 INFO  (jetty-launcher-1204-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62321/solr
   [junit4]   2> 554785 INFO  (zkConnectionManagerCallback-1240-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554785 INFO  (zkCallback-1220-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 554786 DEBUG (OverseerAutoScalingTriggerThread-72065543657357320-127.0.0.1:62323_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:62324_solr, 127.0.0.1:62323_solr]
   [junit4]   2> 554787 DEBUG (ScheduledTrigger-2098-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 554797 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 554798 INFO  (zkConnectionManagerCallback-1246-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554800 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 554800 INFO  (zkConnectionManagerCallback-1248-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554802 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:62321/solr ready
   [junit4]   2> 554802 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 554826 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 554835 DEBUG (OverseerStateUpdate-72065543657357320-127.0.0.1:62323_solr-n_0000000000) [n:127.0.0.1:62323_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:62325_solr"} current state version: 0
   [junit4]   2> 554837 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 554837 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62325_solr
   [junit4]   2> 554840 INFO  (zkCallback-1245-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 554850 INFO  (zkCallback-1235-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 554850 INFO  (zkCallback-1228-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 554851 INFO  (zkCallback-1247-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 554857 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62323.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554861 INFO  (zkCallback-1220-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 554870 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62323.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554873 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62323.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554874 INFO  (jetty-launcher-1204-thread-3) [n:127.0.0.1:62323_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-001/tempDir-001/node3/.
   [junit4]   2> 554878 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62324.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554883 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62324.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554883 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62324.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554884 INFO  (jetty-launcher-1204-thread-2) [n:127.0.0.1:62324_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-001/tempDir-001/node2/.
   [junit4]   2> 554894 INFO  (zkConnectionManagerCallback-1256-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 554895 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 554897 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:62321/solr ready
   [junit4]   2> 554897 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 554927 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62325.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554966 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62325.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554966 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_62325.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@310b9f1b
   [junit4]   2> 554969 INFO  (jetty-launcher-1204-thread-4) [n:127.0.0.1:62325_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-001/tempDir-001/node4/.
   [junit4]   2> 555021 INFO  (jetty-launcher-1204-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62321/solr
   [junit4]   2> 555023 INFO  (zkConnectionManagerCallback-1261-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 555026 INFO  (zkConnectionManagerCallback-1263-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 555040 INFO  (jetty-launcher-1204-thread-1) [n:127.0.0.1:62326_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 555048 INFO  (jetty-launcher-1204-thread-1) [n:127.0.0.1:62326_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 555049 INFO  (jetty-launcher-1204-thread-1) [n:127.0.0.1:62326_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62326_solr
   [junit4]   2> 555049 DEBUG (O

[...truncated too long message...]

ader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 359295 INFO  (jetty-closer-269-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 359296 INFO  (coreCloseExecutor-583-thread-1) [n:127.0.0.1:63995_solr c:raceDeleteReplica_false s:shard1 r:core_node4 x:raceDeleteReplica_false_shard1_replica_n2] o.a.s.c.SolrCore [raceDeleteReplica_false_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@7e184b90
   [junit4]   2> 359296 INFO  (coreCloseExecutor-583-thread-1) [n:127.0.0.1:63995_solr c:raceDeleteReplica_false s:shard1 r:core_node4 x:raceDeleteReplica_false_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.raceDeleteReplica_false.shard1.replica_n2, tag=7e184b90
   [junit4]   2> 359296 INFO  (coreCloseExecutor-583-thread-1) [n:127.0.0.1:63995_solr c:raceDeleteReplica_false s:shard1 r:core_node4 x:raceDeleteReplica_false_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5cc32164: rootName = solr_63995, domain = solr.core.raceDeleteReplica_false.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.raceDeleteReplica_false.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@74652131
   [junit4]   2> 359332 INFO  (coreCloseExecutor-581-thread-1) [n:127.0.0.1:63994_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deletereplica_test.shard1.leader, tag=599ca364
   [junit4]   2> 359337 INFO  (coreCloseExecutor-582-thread-1) [n:127.0.0.1:63993_solr c:raceDeleteReplica_true s:shard1 r:core_node4 x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.raceDeleteReplica_true.shard1.leader, tag=40c28ead
   [junit4]   2> 359342 INFO  (jetty-closer-269-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72065871997763591-127.0.0.1:63994_solr-n_0000000000) closing
   [junit4]   2> 359342 INFO  (OverseerStateUpdate-72065871997763591-127.0.0.1:63994_solr-n_0000000000) [n:127.0.0.1:63994_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:63994_solr
   [junit4]   2> 359343 WARN  (OverseerAutoScalingTriggerThread-72065871997763591-127.0.0.1:63994_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 359346 INFO  (coreCloseExecutor-583-thread-1) [n:127.0.0.1:63995_solr c:raceDeleteReplica_false s:shard1 r:core_node4 x:raceDeleteReplica_false_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.raceDeleteReplica_false.shard1.leader, tag=7e184b90
   [junit4]   2> 359353 INFO  (zkCallback-382-thread-3) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63995_solr
   [junit4]   2> 359356 INFO  (jetty-closer-269-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5776d3a0{/solr,null,UNAVAILABLE}
   [junit4]   2> 359356 INFO  (jetty-closer-269-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1982ec45{/solr,null,UNAVAILABLE}
   [junit4]   2> 359356 INFO  (jetty-closer-269-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 359359 INFO  (jetty-closer-269-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6a42c965{/solr,null,UNAVAILABLE}
   [junit4]   2> 359357 INFO  (jetty-closer-269-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 359359 INFO  (jetty-closer-269-thread-3) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 359361 ERROR (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 359362 INFO  (SUITE-DeleteReplicaTest-seed#[55FCD68D4DE2FE2D]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:63991 63991
   [junit4]   2> 359380 INFO  (Thread-679) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:63991 63991
   [junit4]   2> 359512 WARN  (Thread-679) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/aliases.json
   [junit4]   2> 	13	/solr/collections/raceDeleteReplica_true/terms/shard1
   [junit4]   2> 	10	/solr/configs/conf
   [junit4]   2> 	9	/solr/clusterprops.json
   [junit4]   2> 	8	/solr/collections/deletereplica_test/terms/shard1
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	7	/solr/collections/raceDeleteReplica_false/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	17	/solr/collections/raceDeleteReplica_true/state.json
   [junit4]   2> 	15	/solr/clusterprops.json
   [junit4]   2> 	15	/solr/clusterstate.json
   [junit4]   2> 	7	/solr/collections/raceDeleteReplica_false/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/72065871997763592-127.0.0.1:63995_solr-n_0000000001
   [junit4]   2> 	2	/solr/overseer_elect/election/72065871997763591-127.0.0.1:63994_solr-n_0000000000
   [junit4]   2> 	2	/solr/collections/deletereplica_test/state.json
   [junit4]   2> 	2	/solr/collections/raceDeleteReplica_false/leader_elect/shard1/election/72065871997763609-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/72065871997763605-127.0.0.1:63993_solr-n_0000000004
   [junit4]   2> 	2	/solr/collections/raceDeleteReplica_true/leader_elect/shard1/election/72065871997763599-core_node3-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/live_nodes
   [junit4]   2> 	15	/solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_55FCD68D4DE2FE2D-002
   [junit4]   2> Nov 06, 2018 6:40:25 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{}, maxPointsInLeafNode=81, maxMBSortInHeap=5.990628722140795, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d9700b2), locale=es-SV, timezone=Europe/Budapest
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=3,threads=1,free=145167096,total=266338304
   [junit4]   2> NOTE: All tests run in this JVM: [DeleteReplicaTest, DeleteReplicaTest, DeleteReplicaTest]
   [junit4] Completed [5/5 (3!)] on J1 in 121.44s, 7 tests, 1 failure, 1 error, 4 skipped <<< FAILURES!
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 55FCD68D4DE2FE2D]:
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
   [junit4]   - org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     0.72 ..   239.11 =   238.39s
   [junit4] JVM J1:     0.72 ..   360.51 =   359.79s
   [junit4] Execution time total: 6 minutes
   [junit4] Tests summary: 5 suites, 35 tests, 5 errors, 3 failures, 4 ignored (4 assumptions)

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:1567: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:1092: There were test failures: 5 suites, 35 tests, 5 errors, 3 failures, 4 ignored (4 assumptions) [seed: 55FCD68D4DE2FE2D]

Total time: 6 minutes 2 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   3/5 failed: org.apache.solr.cloud.DeleteReplicaTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=0ddbc4bf953e65ff7af5140b95f8d9edcc245875, workspace=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX
[WARNINGS] Computing warning deltas based on reference build #4914
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2