You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/10/09 15:22:18 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 2099 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/2099/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

4 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([ADC489B7797A736B:2590B66DD7861E93]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.junit.Assert.assertFalse(Assert.java:79)
	at org.apache.solr.cloud.MoveReplicaTest.test(MoveReplicaTest.java:148)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica

Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10010_solr, 127.0.0.1:10008_solr, 127.0.0.1:10012_solr, 127.0.0.1:10009_solr, 127.0.0.1:10011_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"1",   "tlogReplicas":"0",   "autoCreated":"true",   "policy":"c1",   "shards":{"shard1":{       "replicas":{"core_node1":{           "core":"testCreateCollectionAddReplica_shard1_replica_n1",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10009_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"80000000-7fffffff",       "state":"active"}}}

Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10010_solr, 127.0.0.1:10008_solr, 127.0.0.1:10012_solr, 127.0.0.1:10009_solr, 127.0.0.1:10011_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10009_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}
	at __randomizedtesting.SeedInfo.seed([ADC489B7797A736B:2DE4EC9968399BCD]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
	at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 18
live nodes:[127.0.0.1:10010_solr, 127.0.0.1:10008_solr, 127.0.0.1:10012_solr, 127.0.0.1:10009_solr, 127.0.0.1:10011_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10009_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
      "replicas":{"core_node1":{
          "core":"testCreateCollectionAddReplica_shard1_replica_n1",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10009_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"80000000-7fffffff",
      "state":"active"}}}
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
	... 41 more


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testNonRetryableRequests

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([67776E6F83B8CC38:CF99A9FC7DF1B834]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testNonRetryableRequests(CloudSolrClientTest.java:550)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.preferLocalShardsTest

Error Message:
Response was not received from shards on a single node

Stack Trace:
java.lang.AssertionError: Response was not received from shards on a single node
	at __randomizedtesting.SeedInfo.seed([67776E6F83B8CC38:9BBAF6D67D6F5BC2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithShardsPreferenceRules(CloudSolrClientTest.java:471)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.preferLocalShardsTest(CloudSolrClientTest.java:421)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13774 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaTest
   [junit4]   2> 2181300 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/init-core-data-001
   [junit4]   2> 2181300 WARN  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 2181300 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2181306 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2181306 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/tempDir-001
   [junit4]   2> 2181306 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2181306 INFO  (Thread-3612) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2181306 INFO  (Thread-3612) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2181308 ERROR (Thread-3612) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2181409 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:42370
   [junit4]   2> 2181412 INFO  (zkConnectionManagerCallback-9894-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2181417 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1000866d8d50000, likely client has closed socket
   [junit4]   2> 2181421 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2181421 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2181422 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2181422 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2181423 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 2181424 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2181424 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2181424 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2181431 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2181431 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2181431 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 2181431 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e25a8c3{/solr,null,AVAILABLE}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ff276cd{/solr,null,AVAILABLE}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2507b36{/solr,null,AVAILABLE}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@208180c1{/solr,null,AVAILABLE}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d0053e8{HTTP/1.1,[http/1.1]}{127.0.0.1:51690}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@54325f25{HTTP/1.1,[http/1.1]}{127.0.0.1:64785}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-1) [    ] o.e.j.s.Server Started @2183200ms
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-4) [    ] o.e.j.s.Server Started @2183200ms
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64785}
   [junit4]   2> 2181437 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51690}
   [junit4]   2> 2181438 ERROR (jetty-launcher-9891-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2181438 ERROR (jetty-launcher-9891-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-09T14:36:06.920Z
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2181438 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-09T14:36:06.920Z
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1b23ac79{HTTP/1.1,[http/1.1]}{127.0.0.1:36512}
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.e.j.s.Server Started @2183202ms
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36512}
   [junit4]   2> 2181439 ERROR (jetty-launcher-9891-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-09T14:36:06.921Z
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@13bb643d{HTTP/1.1,[http/1.1]}{127.0.0.1:42837}
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.e.j.s.Server Started @2183202ms
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42837}
   [junit4]   2> 2181439 ERROR (jetty-launcher-9891-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2181439 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-09T14:36:06.921Z
   [junit4]   2> 2181444 INFO  (zkConnectionManagerCallback-9896-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2181445 INFO  (zkConnectionManagerCallback-9898-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2181445 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2181445 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2181446 INFO  (zkConnectionManagerCallback-9902-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2181446 INFO  (zkConnectionManagerCallback-9900-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2181446 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2181449 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2182640 INFO  (jetty-launcher-9891-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42370/solr
   [junit4]   2> 2182768 INFO  (zkConnectionManagerCallback-9906-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2182770 DEBUG (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$46/156587264@6a045a53
   [junit4]   2> 2182772 INFO  (zkConnectionManagerCallback-9908-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2182953 INFO  (jetty-launcher-9891-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42370/solr
   [junit4]   2> 2183004 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183004 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183004 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183005 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183005 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183006 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 2183010 DEBUG (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72066831855452166-127.0.0.1:64785_solr-n_0000000000
   [junit4]   2> 2183010 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64785_solr
   [junit4]   2> 2183011 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer Overseer (id=72066831855452166-127.0.0.1:64785_solr-n_0000000000) starting
   [junit4]   2> 2183024 INFO  (zkConnectionManagerCallback-9919-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183026 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183026 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183027 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183027 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183028 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183028 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42370/solr ready
   [junit4]   2> 2183029 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2183029 INFO  (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:64785_solr
   [junit4]   2> 2183033 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64785_solr
   [junit4]   2> 2183034 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183034 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 2183034 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 2183034 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 2183035 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 2183035 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2183036 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2183037 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
   [junit4]   2> 2183037 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2183037 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183037 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 2183037 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2183038 INFO  (zkCallback-9918-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2183041 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 2183044 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:64785_solr]
   [junit4]   2> 2183044 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2183045 DEBUG (OverseerAutoScalingTriggerThread-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2183045 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2183045 DEBUG (ScheduledTrigger-7424-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 2183066 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2183088 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183088 INFO  (zkCallback-9907-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2183106 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64785.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183114 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64785.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183114 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64785.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183115 INFO  (jetty-launcher-9891-thread-1) [n:127.0.0.1:64785_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/tempDir-001/node2/.
   [junit4]   2> 2183147 INFO  (zkConnectionManagerCallback-9914-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183149 DEBUG (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$46/156587264@5401c534
   [junit4]   2> 2183150 INFO  (zkConnectionManagerCallback-9922-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183158 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183159 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183159 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2183159 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183159 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183160 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183160 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 2183163 DEBUG (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72066831855452169-127.0.0.1:51690_solr-n_0000000001
   [junit4]   2> 2183163 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72066831855452166-127.0.0.1:64785_solr-n_0000000000
   [junit4]   2> 2183163 DEBUG (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72066831855452166-127.0.0.1:64785_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 2183164 DEBUG (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:51690_solr as DOWN
   [junit4]   2> 2183165 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2183165 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2183165 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51690_solr
   [junit4]   2> 2183166 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2183166 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:51690_solr"} current state version: 0
   [junit4]   2> 2183166 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:51690_solr
   [junit4]   2> 2183166 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183166 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183166 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183167 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183167 INFO  (zkCallback-9907-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2183209 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183209 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183213 INFO  (zkCallback-9918-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2183209 INFO  (zkCallback-9921-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2183213 INFO  (zkConnectionManagerCallback-9929-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183214 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183215 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183215 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2183215 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183216 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183216 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183216 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42370/solr ready
   [junit4]   2> 2183217 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2183233 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51690.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183241 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51690.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183241 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51690.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2183242 INFO  (jetty-launcher-9891-thread-4) [n:127.0.0.1:51690_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/tempDir-001/node3/.
   [junit4]   2> 2183273 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2183274 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2183924 INFO  (jetty-launcher-9891-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42370/solr
   [junit4]   2> 2183926 INFO  (zkConnectionManagerCallback-9934-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183927 DEBUG (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$46/156587264@129a8a46
   [junit4]   2> 2183929 INFO  (zkConnectionManagerCallback-9936-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183939 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183940 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183940 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2183940 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183940 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183941 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 2183947 DEBUG (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72066831855452172-127.0.0.1:36512_solr-n_0000000002
   [junit4]   2> 2183948 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72066831855452169-127.0.0.1:51690_solr-n_0000000001
   [junit4]   2> 2183948 DEBUG (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72066831855452169-127.0.0.1:51690_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 2183948 DEBUG (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:36512_solr as DOWN
   [junit4]   2> 2183950 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2183950 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2183950 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:36512_solr"} current state version: 0
   [junit4]   2> 2183950 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2183950 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:36512_solr
   [junit4]   2> 2183950 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36512_solr
   [junit4]   2> 2183951 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183951 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183951 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183951 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183951 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2183952 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183952 INFO  (zkCallback-9907-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2183952 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183952 INFO  (zkCallback-9918-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2183952 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183952 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183952 INFO  (zkCallback-9921-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2183952 INFO  (zkCallback-9928-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2183953 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183953 INFO  (zkCallback-9935-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2183977 INFO  (zkConnectionManagerCallback-9943-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2183981 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2183982 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2183982 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2183985 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2183989 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2183990 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2183994 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42370/solr ready
   [junit4]   2> 2183994 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2184011 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36512.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184614 DEBUG (ScheduledTrigger-7424-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 2184614 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2184615 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2184617 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36512.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184617 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36512.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184618 INFO  (jetty-launcher-9891-thread-3) [n:127.0.0.1:36512_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/tempDir-001/node4/.
   [junit4]   2> 2184620 INFO  (jetty-launcher-9891-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42370/solr
   [junit4]   2> 2184627 INFO  (zkConnectionManagerCallback-9948-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184629 DEBUG (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$46/156587264@5d5f6658
   [junit4]   2> 2184633 INFO  (zkConnectionManagerCallback-9950-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184691 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2184691 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184691 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2184692 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2184698 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2184700 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2184703 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 2184710 DEBUG (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72066831855452175-127.0.0.1:42837_solr-n_0000000003
   [junit4]   2> 2184721 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72066831855452172-127.0.0.1:36512_solr-n_0000000002
   [junit4]   2> 2184722 DEBUG (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72066831855452172-127.0.0.1:36512_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 2184726 DEBUG (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:42837_solr as DOWN
   [junit4]   2> 2184733 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2184734 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2184734 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42837_solr
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184735 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2184737 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184737 INFO  (zkCallback-9949-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184738 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184738 INFO  (zkCallback-9935-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184739 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184739 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184739 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184739 INFO  (zkCallback-9921-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184739 INFO  (zkCallback-9907-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184740 INFO  (zkCallback-9918-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184740 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2184740 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184740 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:42837_solr"} current state version: 0
   [junit4]   2> 2184740 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:42837_solr
   [junit4]   2> 2184740 INFO  (zkCallback-9928-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184764 INFO  (zkConnectionManagerCallback-9957-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184765 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2184766 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184766 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2184766 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2184766 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2184767 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2184767 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42370/solr ready
   [junit4]   2> 2184768 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2184789 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42837.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184798 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184798 INFO  (zkCallback-9942-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2184802 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42837.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184802 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42837.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@206bad38
   [junit4]   2> 2184803 INFO  (jetty-launcher-9891-thread-2) [n:127.0.0.1:42837_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaTest_ADC489B7797A736B-001/tempDir-001/node1/.
   [junit4]   2> 2184840 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2184841 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2184861 INFO  (zkConnectionManagerCallback-9960-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184866 INFO  (zkConnectionManagerCallback-9965-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184868 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2184868 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184868 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2184869 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2184869 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2184869 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2184870 INFO  (SUITE-MoveReplicaTest-seed#[ADC489B7797A736B]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42370/solr ready
   [junit4]   2> 2184879 INFO  (qtp71575257-23055) [n:127.0.0.1:36512_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2184879 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 2184880 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 2184880 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 2184880 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{"operation":"overseerstatus"}
   [junit4]   2> 2184881 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
   [junit4]   2> 2184881 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : overseerstatus , {"operation":"overseerstatus"}
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 2184881 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 2184882 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 2184882 DEBUG (OverseerCollectionConfigSetProcessor-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
   [junit4]   2> 2184884 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Completed task:[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 2184889 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 2184889 DEBUG (zkCallback-9935-thread-1) [    ] o.a.s.c.OverseerTaskQueue NodeDataChanged fired on path /overseer/collection-queue-work/qnr-0000000000 state SyncConnected latchEventType null
   [junit4]   2> 2184890 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as completed.
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 2184890 DEBUG (OverseerThreadFactory-7426-thread-1-processing-n:127.0.0.1:64785_solr) [n:127.0.0.1:64785_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{leader=127.0.0.1:64785_solr,overseer_queue_size=0,overseer_work_queue_size=0,overseer_collection_queue_size=2,overseer_operations={am_i_leader={requests=5,errors=0,avgRequestsPerSecond=2.695388540069374,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=1.0806495453998732,medianRequestTime=0.328233,75thPcRequestTime=0.334442,95thPcRequestTime=4.319697,99thPcRequestTime=4.319697,999thPcRequestTime=4.319697},downnode={requests=3,errors=0,avgRequestsPerSecond=1.7465281560001744,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.027918610951800444,medianRequestTime=0.027566,75thPcRequestTime=0.03179,95thPcRequestTime=0.03179,99thPcRequestTime=0.03179,999thPcRequestTime=0.03179}},collection_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=1.617413437089557,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.3485021618976295,medianRequestTime=0.373342,75thPcRequestTime=0.669927,95thPcRequestTime=0.669927,99thPcRequestTime=0.669927,999thPcRequestTime=0.669927},overseerstatus={requests=0,errors=0,avgRequestsPerSecond=0.0,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.0,medianRequestTime=0.0,75thPcRequestTime=0.0,95thPcRequestTime=0.0,99thPcRequestTime=0.0,999thPcRequestTime=0.0}},overseer_queue={},overseer_internal_queue={peek={avgRequestsPerSecond=0.5406578089270494,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.548353,medianRequestTime=0.548353,75thPcRequestTime=0.548353,95thPcRequestTime=0.548353,99thPcRequestTime=0.548353,999thPcRequestTime=0.548353}},collection_queue={peektopn_wait2000={avgRequestsPerSecond=0.5407907720310621,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=1846.822975,medianRequestTime=1846.822975,75thPcRequestTime=1846.822975,95thPcRequestTime=1846.822975,99thPcRequestTime=1846.822975,999thPcRequestTime=1846.822975}}}
   [junit4]   2> 2184891 INFO  (qtp71575257-23055) [n:127.0.0.1:36512_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 2184892 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 2184935 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testFailedMove
   [junit4]   2> 2184937 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2184937 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2184937 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2184937 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2184937 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2184937 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184942 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/aliases.json
   [junit4]   2> 2184943 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184943 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184944 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184944 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/aliases.json
   [junit4]   2> 2184944 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2184944 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2184944 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2184944 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@68510739{/solr,null,AVAILABLE}
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@60fa077c{HTTP/1.1,[http/1.1]}{127.0.0.1:39072}
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.e.j.s.Server Started @2186709ms
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39072}
   [junit4]   2> 2184945 ERROR (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2184945 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-09T14:36:10.427Z
   [junit4]   2> 2184947 INFO  (zkConnectionManagerCallback-9969-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2184948 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2185575 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42370/solr
   [junit4]   2> 2185578 INFO  (zkConnectionManagerCallback-9973-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2185580 DEBUG (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$46/156587264@aa81b7b
   [junit4]   2> 2185582 INFO  (zkConnectionManagerCallback-9975-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2185605 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2185614 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185617 DEBUG (ScheduledTrigger-7424-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 2185618 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2185618 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2185630 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2185635 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2185641 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 2185647 DEBUG (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72066831855452181-127.0.0.1:39072_solr-n_0000000004
   [junit4]   2> 2185648 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72066831855452175-127.0.0.1:42837_solr-n_0000000003
   [junit4]   2> 2185648 DEBUG (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72066831855452175-127.0.0.1:42837_solr-n_0000000003 to know if I could be the leader
   [junit4]   2> 2185648 DEBUG (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:39072_solr as DOWN
   [junit4]   2> 2185658 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 2185658 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 2185658 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2185658 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39072_solr
   [junit4]   2> 2185659 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39072_solr"} current state version: 0
   [junit4]   2> 2185659 DEBUG (OverseerStateUpdate-72066831855452166-127.0.0.1:64785_solr-n_0000000000) [n:127.0.0.1:64785_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39072_solr
   [junit4]   2> 2185659 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185659 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185659 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185660 INFO  (zkCallback-9907-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185660 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185661 INFO  (zkCallback-9928-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185661 INFO  (zkCallback-9949-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185661 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185661 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185661 INFO  (zkCallback-9964-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185661 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185661 INFO  (zkCallback-9964-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185661 INFO  (zkCallback-9942-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185661 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185662 INFO  (zkCallback-9956-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185662 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185662 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185662 INFO  (zkCallback-9935-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185662 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185663 INFO  (zkCallback-9921-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185663 INFO  (zkCallback-9918-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185664 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185664 INFO  (zkCallback-9974-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2185702 INFO  (zkConnectionManagerCallback-9982-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2185709 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
   [junit4]   2> 2185710 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 2185710 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 2185711 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 2185715 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 2185717 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 2185718 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ADC489B7797A736B]) [n:127.0.0.1:39072_solr    ] o.a.s.c.s.i.ZkClientCl

[...truncated too long message...]

tion 1.8.0_172 (64-bit)/cpus=3,threads=1,free=102418552,total=280059904
   [junit4]   2> NOTE: All tests run in this JVM: [SolrExceptionTest, TestDocumentObjectBinder, ConcurrentUpdateSolrClientBuilderTest, FieldValueEvaluatorTest, TestFastWriter, ConcurrentUpdateSolrClientTest, LessThanEqualToEvaluatorTest, TestUpdateRequestCodec, ClientUtilsTest, NaturalLogEvaluatorTest, JdbcDriverTest, EqualToEvaluatorTest, DocumentAnalysisResponseTest, StreamExpressionToExplanationTest, GraphExpressionTest, HyperbolicCosineEvaluatorTest, ShardParamsTest, CoalesceEvaluatorTest, SolrExampleStreamingBinaryTest, TangentEvaluatorTest, UniformDistributionEvaluatorTest, NamedListTest, TestCloudSolrClientConnections, TestConfigSetAdminRequest, OperationsTest, LargeVolumeJettyTest, HyperbolicTangentEvaluatorTest, SelectWithEvaluatorsTest, SolrExampleBinaryTest, TestCloudCollectionsListeners, TestTimeSource, TestNamedListCodec, OrEvaluatorTest, TestZkConfigManager, TestV2Request, JDBCStreamTest, ContentStreamTest, SolrExampleEmbeddedTest, TestPolicy, TestPolicy2, LargeVolumeBinaryJettyTest, LargeVolumeEmbeddedTest, MergeIndexesEmbeddedTest, TestEmbeddedSolrServer, BasicHttpSolrClientTest, CloudSolrClientTest]
   [junit4] Completed [105/157 (1!)] on J1 in 41.05s, 16 tests, 2 failures <<< FAILURES!

[...truncated 44766 lines...]
-ecj-javadoc-lint-src:
    [mkdir] Created dir: /var/tmp/ecj480703848
 [ecj-lint] Compiling 1234 source files to /var/tmp/ecj480703848
 [ecj-lint] Processing annotations
 [ecj-lint] Annotations processed
 [ecj-lint] Processing annotations
 [ecj-lint] No elements to process
 [ecj-lint] invalid Class-Path header in manifest of jar file: /export/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /export/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java (at line 219)
 [ecj-lint] 	return (NamedList<Object>) new JavaBinCodec(resolver).unmarshal(in);
 [ecj-lint] 	                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. ERROR in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/cloud/ElectionContext.java (at line 24)
 [ecj-lint] 	import java.util.HashSet;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.HashSet is never used
 [ecj-lint] ----------
 [ecj-lint] 3. ERROR in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/cloud/ElectionContext.java (at line 26)
 [ecj-lint] 	import java.util.Set;
 [ecj-lint] 	       ^^^^^^^^^^^^^
 [ecj-lint] The import java.util.Set is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 4. ERROR in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/cloud/ZkController.java (at line 113)
 [ecj-lint] 	import org.apache.zookeeper.KeeperException.ConnectionLossException;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.zookeeper.KeeperException.ConnectionLossException is never used
 [ecj-lint] ----------
 [ecj-lint] 5. ERROR in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/cloud/ZkController.java (at line 122)
 [ecj-lint] 	import org.slf4j.MDC;
 [ecj-lint] 	       ^^^^^^^^^^^^^
 [ecj-lint] The import org.slf4j.MDC is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 6. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/cloud/api/collections/RestoreCmd.java (at line 260)
 [ecj-lint] 	throw new SolrException(ErrorCode.BAD_REQUEST, "Unexpected number of replicas, replicationFactor, " +
 [ecj-lint]               Replica.Type.NRT + " or " + Replica.Type.TLOG + " must be greater than 0");
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'repository' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 7. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/handler/loader/JavabinLoader.java (at line 143)
 [ecj-lint] 	new JavaBinCodec() {
 [ecj-lint]       SolrParams params;
 [ecj-lint]       AddUpdateCommand addCmd = null;
 [ecj-lint] 
 [ecj-lint]       @Override
 [ecj-lint]       public List<Object> readIterator(DataInputInputStream fis) throws IOException {
 [ecj-lint]         while (true) {
 [ecj-lint]           Object o = readVal(fis);
 [ecj-lint]           if (o == END_OBJ) break;
 [ecj-lint]           if (o instanceof NamedList) {
 [ecj-lint]             params = ((NamedList) o).toSolrParams();
 [ecj-lint]           } else {
 [ecj-lint]             try {
 [ecj-lint]               if (o instanceof byte[]) {
 [ecj-lint]                 if (params != null) req.setParams(params);
 [ecj-lint]                 byte[] buf = (byte[]) o;
 [ecj-lint]                 contentStreamLoader.load(req, rsp, new ContentStreamBase.ByteArrayStream(buf, null), processor);
 [ecj-lint]               } else {
 [ecj-lint]                 throw new RuntimeException("unsupported type ");
 [ecj-lint]               }
 [ecj-lint]             } catch (Exception e) {
 [ecj-lint]               throw new RuntimeException(e);
 [ecj-lint]             } finally {
 [ecj-lint]               params = null;
 [ecj-lint]               req.setParams(old);
 [ecj-lint]             }
 [ecj-lint]           }
 [ecj-lint]         }
 [ecj-lint]         return Collections.emptyList();
 [ecj-lint]       }
 [ecj-lint] 
 [ecj-lint]     }.unmarshal(in);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 8. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/update/UpdateLog.java (at line 1865)
 [ecj-lint] 	if (exceptionOnExecuteUpdate.get() != null) throw exceptionOnExecuteUpdate.get();
 [ecj-lint] 	                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'proc' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/util/FileUtils.java (at line 50)
 [ecj-lint] 	in = new FileInputStream(src).getChannel();
 [ecj-lint] 	     ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/java/org/apache/solr/util/FileUtils.java (at line 51)
 [ecj-lint] 	out = new FileOutputStream(destination).getChannel();
 [ecj-lint] 	      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 10 problems (4 errors, 6 warnings)

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:633: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:680: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2086: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2125: Compile failed; see the compiler error output for details.

Total time: 103 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 2100 - Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/2100/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

11 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:33624_solr, 127.0.0.1:55491_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/21)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node4":{           "core":"testSimple1_shard1_replica_n3",           "base_url":"https://127.0.0.1:33624/solr",           "node_name":"127.0.0.1:33624_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node12":{           "core":"testSimple1_shard1_replica_n11",           "base_url":"https://127.0.0.1:33624/solr",           "node_name":"127.0.0.1:33624_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node6":{           "core":"testSimple1_shard2_replica_n5",           "base_url":"https://127.0.0.1:33624/solr",           "node_name":"127.0.0.1:33624_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node10":{           "core":"testSimple1_shard2_replica_n9",           "base_url":"https://127.0.0.1:38687/solr",           "node_name":"127.0.0.1:38687_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:33624_solr, 127.0.0.1:55491_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/21)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"testSimple1_shard1_replica_n3",
          "base_url":"https://127.0.0.1:33624/solr",
          "node_name":"127.0.0.1:33624_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node12":{
          "core":"testSimple1_shard1_replica_n11",
          "base_url":"https://127.0.0.1:33624/solr",
          "node_name":"127.0.0.1:33624_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node6":{
          "core":"testSimple1_shard2_replica_n5",
          "base_url":"https://127.0.0.1:33624/solr",
          "node_name":"127.0.0.1:33624_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node10":{
          "core":"testSimple1_shard2_replica_n9",
          "base_url":"https://127.0.0.1:38687/solr",
          "node_name":"127.0.0.1:38687_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:55705_solr, 127.0.0.1:65378_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/22)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node5":{           "core":"testSimple1_shard1_replica_n2",           "base_url":"https://127.0.0.1:55705/solr",           "node_name":"127.0.0.1:55705_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node12":{           "core":"testSimple1_shard1_replica_n11",           "base_url":"https://127.0.0.1:43290/solr",           "node_name":"127.0.0.1:43290_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "core":"testSimple1_shard2_replica_n4",           "base_url":"https://127.0.0.1:55705/solr",           "node_name":"127.0.0.1:55705_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node10":{           "core":"testSimple1_shard2_replica_n9",           "base_url":"https://127.0.0.1:55705/solr",           "node_name":"127.0.0.1:55705_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:55705_solr, 127.0.0.1:65378_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/22)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:55705/solr",
          "node_name":"127.0.0.1:55705_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node12":{
          "core":"testSimple1_shard1_replica_n11",
          "base_url":"https://127.0.0.1:43290/solr",
          "node_name":"127.0.0.1:43290_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:55705/solr",
          "node_name":"127.0.0.1:55705_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node10":{
          "core":"testSimple1_shard2_replica_n9",
          "base_url":"https://127.0.0.1:55705/solr",
          "node_name":"127.0.0.1:55705_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:48571_solr, 127.0.0.1:50853_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/20)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"testSimple1_shard1_replica_n1",           "base_url":"https://127.0.0.1:50853/solr",           "node_name":"127.0.0.1:50853_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node12":{           "core":"testSimple1_shard1_replica_n11",           "base_url":"https://127.0.0.1:46476/solr",           "node_name":"127.0.0.1:46476_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node8":{           "core":"testSimple1_shard2_replica_n6",           "base_url":"https://127.0.0.1:50853/solr",           "node_name":"127.0.0.1:50853_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node10":{           "core":"testSimple1_shard2_replica_n9",           "base_url":"https://127.0.0.1:50853/solr",           "node_name":"127.0.0.1:50853_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:48571_solr, 127.0.0.1:50853_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/20)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "core":"testSimple1_shard1_replica_n1",
          "base_url":"https://127.0.0.1:50853/solr",
          "node_name":"127.0.0.1:50853_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node12":{
          "core":"testSimple1_shard1_replica_n11",
          "base_url":"https://127.0.0.1:46476/solr",
          "node_name":"127.0.0.1:46476_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node8":{
          "core":"testSimple1_shard2_replica_n6",
          "base_url":"https://127.0.0.1:50853/solr",
          "node_name":"127.0.0.1:50853_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node10":{
          "core":"testSimple1_shard2_replica_n9",
          "base_url":"https://127.0.0.1:50853/solr",
          "node_name":"127.0.0.1:50853_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:54325_solr, 127.0.0.1:59361_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/20)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node5":{           "core":"testSimple1_shard1_replica_n2",           "base_url":"https://127.0.0.1:54325/solr",           "node_name":"127.0.0.1:54325_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node12":{           "core":"testSimple1_shard1_replica_n11",           "base_url":"https://127.0.0.1:51107/solr",           "node_name":"127.0.0.1:51107_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "core":"testSimple1_shard2_replica_n4",           "base_url":"https://127.0.0.1:54325/solr",           "node_name":"127.0.0.1:54325_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node10":{           "core":"testSimple1_shard2_replica_n9",           "base_url":"https://127.0.0.1:54325/solr",           "node_name":"127.0.0.1:54325_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:54325_solr, 127.0.0.1:59361_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/20)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:54325/solr",
          "node_name":"127.0.0.1:54325_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node12":{
          "core":"testSimple1_shard1_replica_n11",
          "base_url":"https://127.0.0.1:51107/solr",
          "node_name":"127.0.0.1:51107_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:54325/solr",
          "node_name":"127.0.0.1:54325_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node10":{
          "core":"testSimple1_shard2_replica_n9",
          "base_url":"https://127.0.0.1:54325/solr",
          "node_name":"127.0.0.1:54325_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:51653_solr, 127.0.0.1:55109_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node5":{           "core":"testSimple1_shard1_replica_n2",           "base_url":"https://127.0.0.1:51653/solr",           "node_name":"127.0.0.1:51653_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node12":{           "core":"testSimple1_shard1_replica_n11",           "base_url":"https://127.0.0.1:51653/solr",           "node_name":"127.0.0.1:51653_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node6":{           "core":"testSimple1_shard2_replica_n4",           "base_url":"https://127.0.0.1:51653/solr",           "node_name":"127.0.0.1:51653_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node10":{           "core":"testSimple1_shard2_replica_n9",           "base_url":"https://127.0.0.1:52974/solr",           "node_name":"127.0.0.1:52974_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:51653_solr, 127.0.0.1:55109_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node5":{
          "core":"testSimple1_shard1_replica_n2",
          "base_url":"https://127.0.0.1:51653/solr",
          "node_name":"127.0.0.1:51653_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node12":{
          "core":"testSimple1_shard1_replica_n11",
          "base_url":"https://127.0.0.1:51653/solr",
          "node_name":"127.0.0.1:51653_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node6":{
          "core":"testSimple1_shard2_replica_n4",
          "base_url":"https://127.0.0.1:51653/solr",
          "node_name":"127.0.0.1:51653_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node10":{
          "core":"testSimple1_shard2_replica_n9",
          "base_url":"https://127.0.0.1:52974/solr",
          "node_name":"127.0.0.1:52974_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:57391_solr, 127.0.0.1:58784_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:57391/solr",           "node_name":"127.0.0.1:57391_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:39854/solr",           "node_name":"127.0.0.1:39854_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"down"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:57391/solr",           "node_name":"127.0.0.1:57391_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"active"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:57391/solr",           "node_name":"127.0.0.1:57391_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"active",           "leader":"true"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:57391_solr, 127.0.0.1:58784_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:57391/solr",
          "node_name":"127.0.0.1:57391_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:39854/solr",
          "node_name":"127.0.0.1:39854_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"down"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:57391/solr",
          "node_name":"127.0.0.1:57391_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:57391/solr",
          "node_name":"127.0.0.1:57391_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:60518/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:43206_solr, 127.0.0.1:50792_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:50792/solr",           "node_name":"127.0.0.1:50792_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:32975/solr",           "node_name":"127.0.0.1:32975_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"down"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:50792/solr",           "node_name":"127.0.0.1:50792_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"active"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:50792/solr",           "node_name":"127.0.0.1:50792_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"active",           "leader":"true"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:43206_solr, 127.0.0.1:50792_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:50792/solr",
          "node_name":"127.0.0.1:50792_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:32975/solr",
          "node_name":"127.0.0.1:32975_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"down"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:50792/solr",
          "node_name":"127.0.0.1:50792_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:50792/solr",
          "node_name":"127.0.0.1:50792_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:42396/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:53277_solr, 127.0.0.1:58388_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:53277/solr",           "node_name":"127.0.0.1:53277_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:53277/solr",           "node_name":"127.0.0.1:53277_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"active",           "leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:53277/solr",           "node_name":"127.0.0.1:53277_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:59511/solr",           "node_name":"127.0.0.1:59511_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:53277_solr, 127.0.0.1:58388_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:53277/solr",
          "node_name":"127.0.0.1:53277_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:53277/solr",
          "node_name":"127.0.0.1:53277_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:53277/solr",
          "node_name":"127.0.0.1:53277_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:59511/solr",
          "node_name":"127.0.0.1:59511_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:34322/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:44559_solr, 127.0.0.1:62661_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:44559/solr",           "node_name":"127.0.0.1:44559_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:44559/solr",           "node_name":"127.0.0.1:44559_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"active"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:51320/solr",           "node_name":"127.0.0.1:51320_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"down"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:44559/solr",           "node_name":"127.0.0.1:44559_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"active",           "leader":"true"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:44559_solr, 127.0.0.1:62661_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:44559/solr",
          "node_name":"127.0.0.1:44559_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:44559/solr",
          "node_name":"127.0.0.1:44559_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:51320/solr",
          "node_name":"127.0.0.1:51320_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"down"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:44559/solr",
          "node_name":"127.0.0.1:44559_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:45886/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:52266_solr, 127.0.0.1:54226_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/14)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:54226/solr",           "node_name":"127.0.0.1:54226_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:54226/solr",           "node_name":"127.0.0.1:54226_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"active",           "leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:54226/solr",           "node_name":"127.0.0.1:54226_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:56156/solr",           "node_name":"127.0.0.1:56156_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:52266_solr, 127.0.0.1:54226_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/14)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:54226/solr",
          "node_name":"127.0.0.1:54226_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:54226/solr",
          "node_name":"127.0.0.1:54226_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:54226/solr",
          "node_name":"127.0.0.1:54226_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:56156/solr",
          "node_name":"127.0.0.1:56156_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:63352/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple

Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:43358_solr, 127.0.0.1:51139_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/16)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{         "core_node3":{           "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node3/data/",           "base_url":"http://127.0.0.1:43358/solr",           "node_name":"127.0.0.1:43358_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node3/data/tlog",           "core":"testSimple1_shard1_replica_n1",           "shared_storage":"true",           "state":"active"},         "core_node5":{           "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node5/data/",           "base_url":"http://127.0.0.1:43358/solr",           "node_name":"127.0.0.1:43358_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node5/data/tlog",           "core":"testSimple1_shard1_replica_n2",           "shared_storage":"true",           "state":"active",           "leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node7":{           "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node7/data/",           "base_url":"http://127.0.0.1:43358/solr",           "node_name":"127.0.0.1:43358_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node7/data/tlog",           "core":"testSimple1_shard2_replica_n4",           "shared_storage":"true",           "state":"active",           "leader":"true"},         "core_node8":{           "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node8/data/",           "base_url":"http://127.0.0.1:49607/solr",           "node_name":"127.0.0.1:49607_solr",           "type":"NRT",           "force_set_state":"false",           "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node8/data/tlog",           "core":"testSimple1_shard2_replica_n6",           "shared_storage":"true",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"true",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:43358_solr, 127.0.0.1:51139_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/16)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node3/data/",
          "base_url":"http://127.0.0.1:43358/solr",
          "node_name":"127.0.0.1:43358_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node3/data/tlog",
          "core":"testSimple1_shard1_replica_n1",
          "shared_storage":"true",
          "state":"active"},
        "core_node5":{
          "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node5/data/",
          "base_url":"http://127.0.0.1:43358/solr",
          "node_name":"127.0.0.1:43358_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node5/data/tlog",
          "core":"testSimple1_shard1_replica_n2",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node7/data/",
          "base_url":"http://127.0.0.1:43358/solr",
          "node_name":"127.0.0.1:43358_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node7/data/tlog",
          "core":"testSimple1_shard2_replica_n4",
          "shared_storage":"true",
          "state":"active",
          "leader":"true"},
        "core_node8":{
          "dataDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node8/data/",
          "base_url":"http://127.0.0.1:49607/solr",
          "node_name":"127.0.0.1:49607_solr",
          "type":"NRT",
          "force_set_state":"false",
          "ulogDir":"hdfs://serv1-vm3:51852/data/testSimple1/core_node8/data/tlog",
          "core":"testSimple1_shard2_replica_n6",
          "shared_storage":"true",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"true",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([7584D7D24ABE3B7C:4D37F32C6D4DEFAD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
	at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.eva

[...truncated too long message...]

ting to 127.0.0.1:46965 46965
   [junit4]   2> 682361 WARN  (Thread-626) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	9	/solr/clusterprops.json
   [junit4]   2> 	9	/solr/aliases.json
   [junit4]   2> 	6	/solr/collections/testSimple1/terms/shard2
   [junit4]   2> 	5	/solr/collections/testSimple2/terms/shard2
   [junit4]   2> 	5	/solr/collections/testSimple1/terms/shard1
   [junit4]   2> 	4	/solr/security.json
   [junit4]   2> 	4	/solr/configs/conf
   [junit4]   2> 	3	/solr/collections/testSimple2/terms/shard1
   [junit4]   2> 	3	/solr/collections/testSimple3/terms/shard3
   [junit4]   2> 	3	/solr/collections/testSimple3/terms/shard2
   [junit4]   2> 	3	/solr/collections/testSimple3/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	19	/solr/collections/testSimple3/state.json
   [junit4]   2> 	14	/solr/collections/testSimple1/state.json
   [junit4]   2> 	9	/solr/clusterstate.json
   [junit4]   2> 	7	/solr/clusterprops.json
   [junit4]   2> 	4	/solr/collections/testSimple2/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72071079620116488-127.0.0.1:44559_solr-n_0000000001
   [junit4]   2> 	2	/solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	9	/solr/live_nodes
   [junit4]   2> 	9	/solr/collections
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	2	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 682367 WARN  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 682375 INFO  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 682477 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to serv1-vm3/127.0.0.1:45886) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 682477 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to serv1-vm3/127.0.0.1:45886) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1333979884-127.0.0.1-1539160532215 (Datanode Uuid 5d0c7b98-aab8-4391-81cc-2497f6a9b8f2) service to serv1-vm3/127.0.0.1:45886
   [junit4]   2> 682487 WARN  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 682494 INFO  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 682597 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to serv1-vm3/127.0.0.1:45886) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 682598 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to serv1-vm3/127.0.0.1:45886) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1333979884-127.0.0.1-1539160532215 (Datanode Uuid e579b9d7-6142-491c-adb2-3a82ca1ee5ff) service to serv1-vm3/127.0.0.1:45886
   [junit4]   2> 682617 INFO  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@serv1-vm3:0
   [junit4]   2> 682733 INFO  (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[7584D7D24ABE3B7C]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46965 46965
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_7584D7D24ABE3B7C-003
   [junit4]   2> Oct 10, 2018 8:37:37 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 27 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene80, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@59bde90b), locale=es-HN, timezone=Pacific/Gambier
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=3,threads=2,free=93850760,total=228417536
   [junit4]   2> NOTE: All tests run in this JVM: [AutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest, AutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest]
   [junit4] Completed [10/10 (10!)] on J0 in 136.37s, 1 test, 1 failure <<< FAILURES!
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 7584D7D24ABE3B7C]:
   [junit4]   - org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4]   - org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     0.76 ..   693.09 =   692.33s
   [junit4] JVM J1:     0.75 ..   688.38 =   687.63s
   [junit4] Execution time total: 11 minutes 33 seconds
   [junit4] Tests summary: 10 suites, 10 tests, 10 failures

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1567: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1092: There were test failures: 10 suites, 10 tests, 10 failures [seed: 7584D7D24ABE3B7C]

Total time: 11 minutes 35 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=3629e760113d8faa4b544bffafa3e8b33d2eb404, workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #2098
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2