You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/07/15 08:27:41 UTC

[JENKINS] Lucene-Solr-Tests-master - Build # 2601 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2601/

4 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive

Error Message:
Jetty Connector is not open: -2

Stack Trace:
java.lang.IllegalStateException: Jetty Connector is not open: -2
	at __randomizedtesting.SeedInfo.seed([8C74C868516D5294:9C0E41F6D92EB0C]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getBaseUrl(JettySolrRunner.java:499)
	at org.apache.solr.cloud.MiniSolrCloudCluster.getReplicaJetty(MiniSolrCloudCluster.java:539)
	at org.apache.solr.cloud.LeaderTragicEventTest.corruptLeader(LeaderTragicEventTest.java:100)
	at org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive(LeaderTragicEventTest.java:150)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Error from server at http://127.0.0.1:33704/solr: collection already exists: collection1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33704/solr: collection already exists: collection1
	at __randomizedtesting.SeedInfo.seed([8C74C868516D5294:420F7B2FF913F6C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:37164/solr: Timed out waiting to see all replicas: [hdfsbackuprestore_shard1_1_replica_p0] in cluster state. Last state: DocCollection(hdfsbackuprestore//collections/hdfsbackuprestore/state.json/27)={   "pullReplicas":"1",   "replicationFactor":"2",   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"inactive",       "replicas":{         "core_node3":{           "core":"hdfsbackuprestore_shard1_replica_n1",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node5":{           "core":"hdfsbackuprestore_shard1_replica_n2",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node7":{           "core":"hdfsbackuprestore_shard1_replica_t4",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"TLOG",           "force_set_state":"false"},         "core_node9":{           "core":"hdfsbackuprestore_shard1_replica_p6",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1531636437770460647"},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{         "core_node11":{           "core":"hdfsbackuprestore_shard2_replica_n8",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"hdfsbackuprestore_shard2_replica_n10",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node15":{           "core":"hdfsbackuprestore_shard2_replica_t13",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"hdfsbackuprestore_shard2_replica_p14",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"PULL",           "force_set_state":"false"}}},     "shard1_0":{       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node19":{           "core":"hdfsbackuprestore_shard1_0_replica_n17",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node21":{           "core":"hdfsbackuprestore_shard1_0_replica_n0",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node22":{           "core":"hdfsbackuprestore_shard1_0_replica_t0",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"TLOG",           "force_set_state":"false"},         "core_node23":{           "core":"hdfsbackuprestore_shard1_0_replica_p0",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1531636437770537685"},     "shard1_1":{       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node20":{           "core":"hdfsbackuprestore_shard1_1_replica_n18",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false",           "leader":"true"},         "core_node24":{           "core":"hdfsbackuprestore_shard1_1_replica_n0",           "base_url":"http://127.0.0.1:42745/solr",           "node_name":"127.0.0.1:42745_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node25":{           "core":"hdfsbackuprestore_shard1_1_replica_t0",           "base_url":"http://127.0.0.1:37164/solr",           "node_name":"127.0.0.1:37164_solr",           "state":"active",           "type":"TLOG",           "force_set_state":"false"}},       "stateTimestamp":"1531636437770516623"}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"-1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"1"}

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37164/solr: Timed out waiting to see all replicas: [hdfsbackuprestore_shard1_1_replica_p0] in cluster state. Last state: DocCollection(hdfsbackuprestore//collections/hdfsbackuprestore/state.json/27)={
  "pullReplicas":"1",
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"inactive",
      "replicas":{
        "core_node3":{
          "core":"hdfsbackuprestore_shard1_replica_n1",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node5":{
          "core":"hdfsbackuprestore_shard1_replica_n2",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node7":{
          "core":"hdfsbackuprestore_shard1_replica_t4",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node9":{
          "core":"hdfsbackuprestore_shard1_replica_p6",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1531636437770460647"},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{
        "core_node11":{
          "core":"hdfsbackuprestore_shard2_replica_n8",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"hdfsbackuprestore_shard2_replica_n10",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node15":{
          "core":"hdfsbackuprestore_shard2_replica_t13",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"hdfsbackuprestore_shard2_replica_p14",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"PULL",
          "force_set_state":"false"}}},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node19":{
          "core":"hdfsbackuprestore_shard1_0_replica_n17",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node21":{
          "core":"hdfsbackuprestore_shard1_0_replica_n0",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node22":{
          "core":"hdfsbackuprestore_shard1_0_replica_t0",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node23":{
          "core":"hdfsbackuprestore_shard1_0_replica_p0",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1531636437770537685"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node20":{
          "core":"hdfsbackuprestore_shard1_1_replica_n18",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"},
        "core_node24":{
          "core":"hdfsbackuprestore_shard1_1_replica_n0",
          "base_url":"http://127.0.0.1:42745/solr",
          "node_name":"127.0.0.1:42745_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node25":{
          "core":"hdfsbackuprestore_shard1_1_replica_t0",
          "base_url":"http://127.0.0.1:37164/solr",
          "node_name":"127.0.0.1:37164_solr",
          "state":"active",
          "type":"TLOG",
          "force_set_state":"false"}},
      "stateTimestamp":"1531636437770516623"}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"1"}
	at __randomizedtesting.SeedInfo.seed([8C74C868516D5294:420F7B2FF913F6C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:135)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.component.InfixSuggestersTest.testShutdownDuringBuild

Error Message:
junit.framework.AssertionFailedError: Unexpected wrapped exception type, expected CoreIsClosedException

Stack Trace:
java.util.concurrent.ExecutionException: junit.framework.AssertionFailedError: Unexpected wrapped exception type, expected CoreIsClosedException
	at __randomizedtesting.SeedInfo.seed([8C74C868516D5294:53F9AAD76F0407F6]:0)
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.solr.handler.component.InfixSuggestersTest.testShutdownDuringBuild(InfixSuggestersTest.java:130)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: junit.framework.AssertionFailedError: Unexpected wrapped exception type, expected CoreIsClosedException
	at org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2696)
	at org.apache.solr.handler.component.InfixSuggestersTest.lambda$testShutdownDuringBuild$3(InfixSuggestersTest.java:121)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more
Caused by: java.lang.RuntimeException: Exception during query
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:917)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:877)
	at org.apache.solr.handler.component.InfixSuggestersTest.lambda$null$2(InfixSuggestersTest.java:122)
	at org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2688)
	... 6 more
Caused by: org.apache.solr.common.SolrException: SolrCoreState already closed.
	at org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:186)
	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:121)
	at org.apache.solr.update.SolrCoreState.getTragicException(SolrCoreState.java:198)
	at org.apache.solr.core.CoreContainer.checkTragicException(CoreContainer.java:1812)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:337)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:319)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:891)
	... 9 more




Build Log:
[...truncated 12405 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
   [junit4]   2> 432339 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/init-core-data-001
   [junit4]   2> 432340 WARN  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 432340 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 432358 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 432491 WARN  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 432542 WARN  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 432544 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 432703 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_37937_hdfs____.ozem1p/webapp
   [junit4]   2> 439248 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:37937
   [junit4]   2> 439721 WARN  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 439722 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 439780 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35963_datanode____.xuqghx/webapp
   [junit4]   2> 440677 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35963
   [junit4]   2> 441908 WARN  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 441921 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 442036 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40885_datanode____.km4uhs/webapp
   [junit4]   2> 442462 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-001/hdfsBaseDir/data/data2/]]  heartbeating to lucene2-us-west.apache.org/127.0.0.1:40951) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 442693 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x31a516a75e3fdc: from storage DS-4deaed49-f1a8-4861-bf52-f3cc4d7d9969 node DatanodeRegistration(127.0.0.1:41902, datanodeUuid=092c08b3-b748-402a-ab5a-69f99fc334e7, infoPort=36167, infoSecurePort=0, ipcPort=46709, storageInfo=lv=-56;cid=testClusterID;nsid=1080494504;c=0), blocks: 0, hasStaleStorage: true, processing time: 20 msecs
   [junit4]   2> 442693 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x31a516a75e3fdc: from storage DS-1f40a41f-94b8-4d79-9020-669cfae3a864 node DatanodeRegistration(127.0.0.1:41902, datanodeUuid=092c08b3-b748-402a-ab5a-69f99fc334e7, infoPort=36167, infoSecurePort=0, ipcPort=46709, storageInfo=lv=-56;cid=testClusterID;nsid=1080494504;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 443509 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40885
   [junit4]   2> 444492 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-001/hdfsBaseDir/data/data4/]]  heartbeating to lucene2-us-west.apache.org/127.0.0.1:40951) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 444552 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x31a51718fdf256: from storage DS-22e28d6b-1280-449b-9a4e-f2e2d05e2e76 node DatanodeRegistration(127.0.0.1:35680, datanodeUuid=dbbf1987-5d71-4a3d-aedd-3b90fe7639bc, infoPort=33640, infoSecurePort=0, ipcPort=46710, storageInfo=lv=-56;cid=testClusterID;nsid=1080494504;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 444552 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x31a51718fdf256: from storage DS-13394bf0-4fa9-4d36-813f-732659ec9105 node DatanodeRegistration(127.0.0.1:35680, datanodeUuid=dbbf1987-5d71-4a3d-aedd-3b90fe7639bc, infoPort=33640, infoSecurePort=0, ipcPort=46710, storageInfo=lv=-56;cid=testClusterID;nsid=1080494504;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 444699 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002
   [junit4]   2> 444699 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 444700 INFO  (Thread-1091) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 444700 INFO  (Thread-1091) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 444721 ERROR (Thread-1091) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 444800 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:33978
   [junit4]   2> 444867 INFO  (zkConnectionManagerCallback-432-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 444921 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 444941 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 445030 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 445030 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 445030 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 445030 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f1904d9{/solr,null,AVAILABLE}
   [junit4]   2> 445031 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@42b273a9{HTTP/1.1,[http/1.1]}{127.0.0.1:37164}
   [junit4]   2> 445087 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 445087 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 445087 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 445087 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66008550{/solr,null,AVAILABLE}
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2e5ce6c9{HTTP/1.1,[http/1.1]}{127.0.0.1:42745}
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.e.j.s.Server Started @445126ms
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42745}
   [junit4]   2> 445088 ERROR (jetty-launcher-429-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 445088 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-15T06:33:12.110Z
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.e.j.s.Server Started @445068ms
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37164}
   [junit4]   2> 445089 ERROR (jetty-launcher-429-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 445089 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-15T06:33:12.111Z
   [junit4]   2> 445147 INFO  (zkConnectionManagerCallback-434-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 445161 INFO  (zkConnectionManagerCallback-436-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 445169 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 445180 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 445183 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 445206 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 445568 INFO  (jetty-launcher-429-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33978/solr
   [junit4]   2> 445656 INFO  (zkConnectionManagerCallback-440-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 445721 INFO  (zkConnectionManagerCallback-442-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 446007 INFO  (jetty-launcher-429-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33978/solr
   [junit4]   2> 446009 INFO  (zkConnectionManagerCallback-448-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 446033 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10340e727940005, likely client has closed socket
   [junit4]   2> 446056 INFO  (zkConnectionManagerCallback-450-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 446158 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 446159 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42745_solr
   [junit4]   2> 446178 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.Overseer Overseer (id=72973380513693702-127.0.0.1:42745_solr-n_0000000000) starting
   [junit4]   2> 446245 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 446246 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37164_solr
   [junit4]   2> 446247 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://lucene2-us-west.apache.org:40951/solr,solr.hdfs.confdir=}}
   [junit4]   2> 446247 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://lucene2-us-west.apache.org:40951/solr,solr.hdfs.confdir=}}
   [junit4]   2> 446389 INFO  (zkCallback-449-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 446400 INFO  (zkCallback-441-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 446432 INFO  (zkConnectionManagerCallback-457-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 446510 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 446511 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33978/solr ready
   [junit4]   2> 446530 INFO  (zkConnectionManagerCallback-462-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 446531 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 446547 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33978/solr ready
   [junit4]   2> 446562 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 446631 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 446650 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 446650 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42745_solr
   [junit4]   2> 446687 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://lucene2-us-west.apache.org:40951/solr,solr.hdfs.confdir=}}
   [junit4]   2> 446687 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://lucene2-us-west.apache.org:40951/solr,solr.hdfs.confdir=}}
   [junit4]   2> 446701 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 446701 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 446702 INFO  (jetty-launcher-429-thread-1) [n:127.0.0.1:37164_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/.
   [junit4]   2> 446705 INFO  (zkCallback-449-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 446729 INFO  (zkCallback-441-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 446933 INFO  (zkCallback-456-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 446933 INFO  (zkCallback-461-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 446969 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 447070 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 447143 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 447143 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 447161 INFO  (jetty-launcher-429-thread-2) [n:127.0.0.1:42745_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/.
   [junit4]   2> 447349 INFO  (zkConnectionManagerCallback-466-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 447366 INFO  (zkConnectionManagerCallback-471-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 447376 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 447377 INFO  (SUITE-TestHdfsCloudBackupRestore-seed#[8C74C868516D5294]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33978/solr ready
   [junit4]   2> 447581 INFO  (TEST-TestHdfsCloudBackupRestore.test-seed#[8C74C868516D5294]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 447608 INFO  (qtp1148085066-2547) [n:127.0.0.1:37164_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&property.customKey=customValue&collection.configName=conf1&maxShardsPerNode=-1&name=hdfsbackuprestore&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 447625 INFO  (OverseerThreadFactory-595-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection hdfsbackuprestore
   [junit4]   2> 447626 WARN  (OverseerThreadFactory-595-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection hdfsbackuprestore is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 447761 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"hdfsbackuprestore_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42745/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447785 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"hdfsbackuprestore_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37164/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447800 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"hdfsbackuprestore_shard1_replica_t4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42745/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447815 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"hdfsbackuprestore_shard1_replica_p6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37164/solr",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447832 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"hdfsbackuprestore_shard2_replica_n8",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42745/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447835 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"hdfsbackuprestore_shard2_replica_n10",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37164/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447883 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"hdfsbackuprestore_shard2_replica_t13",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42745/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 447901 INFO  (OverseerStateUpdate-72973380513693702-127.0.0.1:42745_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"hdfsbackuprestore",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"hdfsbackuprestore_shard2_replica_p14",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37164/solr",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 448141 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr    x:hdfsbackuprestore_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node3&name=hdfsbackuprestore_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 448158 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr    x:hdfsbackuprestore_shard1_replica_p6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=PULL&property.customKey=customValue&coreNodeName=core_node9&name=hdfsbackuprestore_shard1_replica_p6&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 448158 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr    x:hdfsbackuprestore_shard1_replica_t4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node7&name=hdfsbackuprestore_shard1_replica_t4&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 448158 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr    x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 448158 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr    x:hdfsbackuprestore_shard2_replica_n8] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node11&name=hdfsbackuprestore_shard2_replica_n8&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 448159 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr    x:hdfsbackuprestore_shard2_replica_n10] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node12&name=hdfsbackuprestore_shard2_replica_n10&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 448226 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr    x:hdfsbackuprestore_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node5&name=hdfsbackuprestore_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 448249 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr    x:hdfsbackuprestore_shard2_replica_t13] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node15&name=hdfsbackuprestore_shard2_replica_t13&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 448249 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr    x:hdfsbackuprestore_shard2_replica_p14] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=PULL&property.customKey=customValue&coreNodeName=core_node16&name=hdfsbackuprestore_shard2_replica_p14&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 449325 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 449362 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 449610 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 449646 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n10] Schema name=minimal
   [junit4]   2> 449661 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 449663 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 449663 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n2' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 449664 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n2' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 449698 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 449698 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n10' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 449698 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n10' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 449735 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_p6] Schema name=minimal
   [junit4]   2> 449737 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 449737 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_p6' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 449760 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 449754 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_p6' (registry 'solr.core.hdfsbackuprestore.shard1.replica_p6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 449975 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 449945 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 449982 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n10] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/hdfsbackuprestore_shard2_replica_n10], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/./hdfsbackuprestore_shard2_replica_n10/data/]
   [junit4]   2> 449987 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 450010 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450012 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 450023 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450023 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_p6] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_p6], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_p6/data/]
   [junit4]   2> 450044 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_n2/data/]
   [junit4]   2> 450056 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_t13] Schema name=minimal
   [junit4]   2> 450058 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 450101 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 450113 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 450113 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_t13' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 450114 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_t13' (registry 'solr.core.hdfsbackuprestore.shard2.replica_t13') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 450114 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450114 INFO  (qtp637574897-2554) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node15 x:hdfsbackuprestore_shard2_replica_t13] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_t13] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_t13], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_t13/data/]
   [junit4]   2> 450136 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 450136 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n1' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 450137 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n1' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 450137 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450139 INFO  (qtp637574897-2556) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/hdfsbackuprestore_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/./hdfsbackuprestore_shard1_replica_n1/data/]
   [junit4]   2> 450378 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n8] Schema name=minimal
   [junit4]   2> 450381 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_p14] Schema name=minimal
   [junit4]   2> 450383 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 450383 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_p14' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 450395 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 450395 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n8' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 450396 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n8' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 450396 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450430 INFO  (qtp637574897-2551) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n8] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_n8], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_n8/data/]
   [junit4]   2> 450447 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_t4] Schema name=minimal
   [junit4]   2> 450449 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 450449 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_t4' using configuration from collection hdfsbackuprestore, trusted=true
   [junit4]   2> 450453 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_p14' (registry 'solr.core.hdfsbackuprestore.shard2.replica_p14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 450453 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450453 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_p14] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/hdfsbackuprestore_shard2_replica_p14], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node1/./hdfsbackuprestore_shard2_replica_p14/data/]
   [junit4]   2> 450482 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_t4' (registry 'solr.core.hdfsbackuprestore.shard1.replica_t4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3019eb97
   [junit4]   2> 450482 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 450482 INFO  (qtp637574897-2552) [n:127.0.0.1:42745_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_t4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/hdfsbackuprestore_shard1_replica_t4], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_8C74C868516D5294-001/tempDir-002/node2/./hdfsbackuprestore_shard1_replica_t4/data/]
   [junit4]   2> 451307 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 451307 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 451308 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 451308 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 451382 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.s.SolrIndexSearcher Opening [Searcher@bb3ba62[hdfsbackuprestore_shard2_replica_n10] main]
   [junit4]   2> 451393 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 451393 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 451394 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 451446 INFO  (searcherExecutor-602-thread-1-processing-n:127.0.0.1:37164_solr x:hdfsbackuprestore_shard2_replica_n10 c:hdfsbackuprestore s:shard2 r:core_node12) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_n10] Registered new searcher Searcher@bb3ba62[hdfsbackuprestore_shard2_replica_n10] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 451453 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606037168167321600
   [junit4]   2> 451632 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 451632 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 451641 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 451641 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 451644 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.s.SolrIndexSearcher Opening [Searcher@18044f54[hdfsbackuprestore_shard1_replica_p6] main]
   [junit4]   2> 451727 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 451727 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 451787 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 451787 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 452064 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5de270b4[hdfsbackuprestore_shard1_replica_n2] main]
   [junit4]   2> 452081 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.s.SolrIndexSearcher Opening [Searcher@6169ad6e[hdfsbackuprestore_shard2_replica_p14] main]
   [junit4]   2> 452102 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.ZkShardTerms Successful update of terms at /collections/hdfsbackuprestore/terms/shard2 to Terms{values={core_node12=0}, version=0}
   [junit4]   2> 452104 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 452140 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 452141 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 452141 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 452264 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 452393 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 452395 INFO  (searcherExecutor-603-thread-1-processing-n:127.0.0.1:37164_solr x:hdfsbackuprestore_shard1_replica_p6 c:hdfsbackuprestore s:shard1 r:core_node9) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_p6] Registered new searcher Searcher@18044f54[hdfsbackuprestore_shard1_replica_p6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 452395 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 452412 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.ZkController hdfsbackuprestore_shard1_replica_p6 starting background replication from leader
   [junit4]   2> 452412 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 452430 INFO  (qtp1148085066-2542) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_n10] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=3 found=1 timeoutin=9979ms
   [junit4]   2> 452430 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 452430 INFO  (qtp1148085066-2549) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606037169191780352
   [junit4]   2> 452433 INFO  (qtp1148085066-2546) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard2 r:core_node16 x:hdfsbackuprestore_shard2_replica_p14] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 452505 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_replica_p6] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 452505 INFO  (qtp1148085066-2543) [n:127.0.0.1:37164_solr c:hdfsbackuprestore s:shard1 r:core_node9 x:hdfsbackuprestore_shard1_rep

[...truncated too long message...]

ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2011235927
     [copy] Copying 241 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2011235927
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2011235927

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 73ms :: artifacts dl 8ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 196 minutes 54 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-Tests-master - Build # 2602 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2602/

1 tests failed.
FAILED:  org.apache.lucene.document.TestLatLonShape.testBasicIntersects

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([95119B56BB36BC74:A26C68A387A11F0A]:0)
	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
	at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:221)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
	at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
	at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.writeField(AssertingPointsFormat.java:137)
	at org.apache.lucene.index.PointValuesWriter.flush(PointValuesWriter.java:183)
	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:215)
	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:141)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:470)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:554)
	at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:413)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:514)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1601)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1220)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
	at org.apache.lucene.document.TestLatLonShape.testBasicIntersects(TestLatLonShape.java:93)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)




Build Log:
[...truncated 9599 lines...]
   [junit4] Suite: org.apache.lucene.document.TestLatLonShape
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLatLonShape -Dtests.method=testBasicIntersects -Dtests.seed=95119B56BB36BC74 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=it -Dtests.timezone=America/Araguaina -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   33.5s J2 | TestLatLonShape.testBasicIntersects <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([95119B56BB36BC74:A26C68A387A11F0A]:0)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    > 	at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:221)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.writeField(AssertingPointsFormat.java:137)
   [junit4]    > 	at org.apache.lucene.index.PointValuesWriter.flush(PointValuesWriter.java:183)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:215)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:141)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:470)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:554)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:413)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:514)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1601)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1220)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
   [junit4]    > 	at org.apache.lucene.document.TestLatLonShape.testBasicIntersects(TestLatLonShape.java:93)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=522, maxMBSortInHeap=5.851519002552058, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4b30ea34), locale=it, timezone=America/Araguaina
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=324557208,total=411041792
   [junit4]   2> NOTE: All tests run in this JVM: [TestTessellator, TestFieldCacheTermsFilter, TestIntervals, TestLatLonPointQueries, TestHalfFloatPoint, TestLatLonShape]
   [junit4] Completed [21/24 (1!)] on J2 in 33.52s, 2 tests, 1 error, 1 skipped <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J2: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/sandbox/test/temp/junit4-J2-20180715_095917_1981508393259200715697.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/heapdumps/java_pid23461.hprof ...
   [junit4] Heap dump file created [439222575 bytes in 2.085 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 14370 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:633: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid23461.hprof

Total time: 176 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2601 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any